Information processing apparatus and information processing method

ABSTRACT

Section information of a traveling route is provided appropriately to a driver. 
     Traveling route information and traffic information relating to the traveling route are acquired, and on the basis of the information, a driver intervention requiring section and an automatic driving available section of the traveling route are displayed on a reach prediction time axis from a current point on an instrument panel, a tablet, or the like. For example, the driver intervention requiring section includes a manual driving section, a takeover section from automatic driving to manual driving, and a cautious traveling section from the automatic driving.

TECHNICAL FIELD

The present technology relates to an information processing apparatusand an information processing method, and particularly to an informationprocessing apparatus and so forth for displaying information relating toa traveling route.

BACKGROUND ART

In recent years, development of a system is being proceeded whichautomatically performs control and support of traveling of a vehicleindependently of an operation of a driver in order to achieveimprovement in safety of vehicle traveling, reduction in load on thedriver, and so forth. In the future, capability of execution of asecondary task in a vehicle during automatic driving is expected.However, as a present road infrastructure, it is assumed thatintroduction of an environmental infrastructure as a traveling route inwhich a driver intervention requiring section that is a sectionrequiring a so-called manual driving, and an automatic driving availablesection are mixed in a speckled manner, progresses. Therefore, in orderto allow a driver to perform execution of a secondary task favorably, itis necessary to appropriately provide section information such as manualor automatic driving sections on the traveling route, and it isnecessary for the latest information for each progress section duringtraveling to be ready for a change of the situation that changes everymoment. Further, section end points for such automatic driving may notnecessarily be fixed.

For example, PTL 1 discloses a technology of displaying, in regard toeach of a plurality of risks on a road on which an own vehicle is totravel, a degree of the risk. Meanwhile, for example, PTL 2 discloses atechnology of displaying, in a case where a driver is to be urged tostart manual driving during automatic driving, that the vehicle is in asituation in which manual driving must be started on the screen of aportable terminal to notify the driver whose consciousness isconcentrated on the portable terminal.

CITATION LIST Patent Literature [PTL 1]

Japanese Patent Laid-Open No. 2016-139204

[PTL 2]

Japanese Patent Laid-Open No. 2016-090274

SUMMARY Technical Problems

Although maturity of a vehicle environment recognition technology forperforming automatic driving, development of map information of a roadtraveling environment, and so forth are raising the momentum ofutilizing automatic driving of a vehicle in an actual world space atpresent, actually it is difficult to implement a closed-track-equippedenvironmental space in which automatic driving is partly available likea subway in any continuous section in which a vehicle can travel.Actually, the continuous section actually is a traveling section inwhich roads of driving levels that allow any automatic driving and roadsthat do not allow any automatic driving are connected at random.Therefore, it is anticipated that road environments are configured fromsections in which automatic driving is available to sections that areunder driver supervision or essentially require manual drivingintervention or the like. Further, in the case of traveling on suchmixed road sections, if return to manual driving that is requested tothe driver upon crossing sections is not performed precisely, thevehicle cannot continue traveling, and therefore, it becomes necessaryfor the vehicle to emergently stop. Therefore, there is a significantsubject that utilization of the social infrastructure as a road isdisturbed. Furthermore, the spread of automatic driving faces such acontradiction that it cannot be satisfied unless an environment isimplemented in which automatic driving can be achieved continuously inall sections in the society as a whole and environment maintenance iscomplete all the time without obstructing traveling.

Therefore, a concept that only a closed environment is utilized forfully automatic driving or a proposal to emergently stop a vehicle in acase where appropriate takeover from automatic driving to manual drivingcannot be performed are provided. Actually, however, if emergentlystopping vehicles overflow in a road infrastructure, this inducestraffic jams in a road environment, or increase of such emergentlystopping vehicles induces an accident that has not been occurred in thepast. Thus, a problem that normal social activities are inhibited occursas a new problem, and after all, a method for widely spreading automaticdriving has not been found out.

For example, even if driving of the level 3 of automatic driving that iswidely discussed currently is introduced just simply into the society ingeneral roads or exclusive road sections, for example, in order tosatisfy the driving without a negative social impact such as jams, it isnecessary that the environment in which a vehicle can travel at thelevel 3 on the road all the time is maintained 100% in the section andthe driver returns with certainty at an end spot. Further, during theperiod, the driver is demanded to always take the responsibility formonitoring without being directly engaged in driving and keep the stateunder tense state with attention. In short, if human ergonomic andpsychological aspects of the human are taken into consideration, thereis a subject that long term utilization according to the applicableconcept is not realistic, and this is a subject to socially introduceautomatic driving in a wide area, and a solution of the subject isdemanded.

The object of the present technology is to normally provide, in order todeal with the subjects described above, section information of atraveling route to the driver before the vehicle approaches a returnrequiring section in various road environments by actively adjusting thecontrol in response to a state of the driver, a traveling property ofthe vehicle, information regarding a road environment, and so forth toprovide appropriate necessary intervention information to the driver inadvance thereby to seamlessly achieve section passage with a highprobability without stopping the vehicle. For introduction of automaticdriving, since the subject that a precise driver return technology uponcrossing these sections cannot be implemented is not successfully solvedas yet, the section whose social introduction is possible is veryrestrictive such as a specific expressway section or an establishedsection.

Solution to Problems

The concept of the present technology resides in an informationprocessing apparatus including an information acquisition unitconfigured to acquire traveling route information and trafficinformation relating to the traveling route, and a display controllingunit configured to update and display a driver intervention requiringsection, traveling property on the traveling route of an own vehicle,and an automatic driving available section of the traveling route on areach prediction time axis from a point at which the own vehicle iscurrently traveling so as to be intuitively recognizable on a displaydevice all the time on the basis of the traveling route information andthe traffic information.

In the present technology, the information acquisition unit acquirestraveling route information and traffic information relating to thetraveling route. Further, according to active decision state of a returndelay characteristic of a particular driver in a particular vehicletaking the traveling property of the own vehicle at the time, in theweather, and the like, into consideration, the display controlling unitdisplays a driver intervention requiring section and an automaticdriving available section of the traveling route on a reach predictiontime axis from the current point on the display device on the basis ofthe traveling route information and the traffic information. Forexample, the display device may be a display device provided in aportable terminal, and the information processing apparatus may furtherinclude a communication unit configured to communicate with the portableterminal.

Although the technology for performing specification notification likethat of PTL 1 or PTL 2 is known, since the technology does not performoptimization notification of return to manual driving from automaticdriving taking a return delay characteristic into account in response toan environmental condition change, supposed by popularization ofautomatic driving, of a traveling route that changes every moment, aloading weight and a braking capacity of an own vehicle, further, areturn characteristic of a driver, and still further, a state of thedriver, the notification performed every time is performed at a timingdifferent from a timing at which it is required by actual driving, andtherefore, the necessity for actual return responsive to thenotification gradually becomes ambiguous. Different from a notificationat passage of a prescribed point for each predetermined passage pointsupposed to be provided to such drivers in the past, informationnecessary for driving intervention return to a driver is provided moreaccurately at an appropriate timing and with an accurate time sense tothe driver. Therefore, optimization of the notification is achieved inthat it is not too early and not too late. As a result, a main vehicletraveling along a road can appropriately perform takeover from automaticdriving to manual driving appropriately even if an environmental changeand the like occurs every moment, and this decreases the burden on theroad infrastructure caused by failure in takeover. Thus, even if anautomatic driving vehicle is introduced, operational failure of thesocial road infrastructure is not caused. The operational failure of theinfrastructure described here refers to all that, in a case where avehicle in which takeover from automatic driving to manual driving isnot performed normally appears frequently, in a road section in whichthe vehicle traffic bandwidth of the road infrastructure is narrow, ifmany emergently slowdown vehicles or stopping vehicles appear, the flowof vehicles in the applicable road section is slowed down or disturbed,resulting in failure to maintain a normal traffic amount.

For example, in a case where route setting is performed and traveling isperformed, on a map to be presented to the driver, the driverintervention requiring section may include a manual driving section, atakeover section from automatic driving to manual driving, and acautious traveling section from the automatic driving. In this case, forexample, the display controlling unit may display the automatic drivingavailable section in a first color, display the manual driving sectionin a second color, and display the takeover section and the cautioustraveling section in a third color. This makes it possible for thedriver to visually observe a manual driving section, a takeover sectionfrom automatic driving to manual driving, a cautious traveling sectionfrom the automatic driving and an automatic driving available section ona traveling route. However, in a case where the driver does not have asense of an average passage speed in each section normally utilizing theapplicable route section, if only section display is made on the map,then the temporal sense until a point at which driving intervention isrequired is reached is entrusted to rules of thumb, and it is difficultto intuitively know a time margin in regard to what can be done till thetakeover point.

Therefore, it is necessary for the driver to preferentially put a matterof a return point normally to consciousness during a work of a secondarytask, which is a maximum benefit of automatic driving in the past, andthis makes an obstruction to execution of the secondary task.Simultaneously, when an increase of attention is demanded upon return asa result of the fact that continued attention is forced for anunnecessary period, if the tension state with attention for a longperiod of time in secondary task activities during the period continues,the perceptual cognitive sensation is paralyzed and the attention forreturn becomes ambiguous. However, where the reach prediction time fromeach current point to each takeover point is always updated anddisplayed intuitively as a time axis, by utilization of both ofexecution of a secondary task with a secure feeling and timelynotification, the return point is always known suitably in advance, andrecognition of a takeover point can be performed easily and timely.

Further, for example, the display controlling unit may display a firstsection from the current point to a first point on a first time axis,display a second section from the first point to a second point in atime axis that sequentially changes from the first time axis to a secondtime axis reduced at a predetermined ratio with respect to the firsttime axis, and display a third section from the second point to a thirdpoint on the second time axis. This makes it possible for the driver toparticularly know section information nearest in time in a limiteddisplay space and know section information farther in time.

In this case, for example, the display controlling unit may display thefirst section with a first width, display the second section with awidth that sequentially changes from the first width to a second widthsmaller than the first width, and display the third section with thesecond width. This makes it possible for the driver to visually andintuitively recognize a degree of reduction of the time axis in thesecond section and the third section with respect to the first section.

Further, in this case, the driving vehicle intervention requiringsection in the third section may be displayed with a fixed time lengtheven if the driving vehicle intervention requiring section actually isequal to or smaller than the fixed time length. This makes it possibleto display, in the third section in which the time axis is reduced by agreat amount, the driving vehicle intervention requiring section of ashort period so as to be easily recognizable by the driver.

Further, for example, the display controlling unit may further displayinformation relating to a point designated in each of the displayedsections. This makes it possible for the driver to designate anarbitrary point in each section and acquire information relating to thepoint.

Further, for example, the display controlling unit may display thedriving vehicle intervention requiring section that appears newly so asto be identifiable from the existing driving vehicle interventionrequiring section. In this case, for example, the newly appearingdriving vehicle intervention requiring section is flickering displayedor displayed in a different color. This makes it possible for the driverto easily recognize the newly appearing driving vehicle interventionrequiring section and explicitly grasp a plan to be coped with orchanged in regard to traveling planning having been planned before theappearance of the additional event.

Further, for example, when the driver intervention requiring sectioncomes in a range of a fixed period of time from the current point, thedriver intervention requiring section, the display controlling unit mayput into an emphatically displayed state. In this case, for example,flickering display, display in a different color, illusion display bywhich the moving speed looks higher than an actual speed or, forexample, wave display is used. This makes it possible for the driver toeasily recognize that a driver intervention requiring section comes in arange of a fixed period of time from the current point. The flickeringdisplay of the dynamic display acts to stimulate the human dynamicvisual acuity and is a method that uses means useful for calling forattention.

Further, for example, the display controlling unit may display images inthe sections in parallel to a work window. This makes it possible forthe driver, who is performing a work with the work window, to easilyrecognize a driver intervention requiring section and an automaticdriving available section of the traveling route on the reach predictiontime axis from the current point. In a case where a secondary task isperformed using the same equipment, desirably the equipment is a displayapparatus that can execute multitasks, and it may be displayed as a subwindow in the case of a tablet terminal or a smartphone terminal or maybe a video player, a game terminal, a video conference system, or thelike.

In this manner, in the present technology, a driver interventionrequiring section and an automatic driving available section of atraveling route are displayed on a reach prediction time axis from thecurrent point on a display device on the basis of traveling routeinformation and traffic information. Therefore, section information fortraveling for the time being of the traveling route can be providedappropriately to the driver. The display update method for them needs tobe performed by the driver who accurately grasps the section approachinformation and has consciousness. However, on the other hand, ifinformation is always updated and displayed in the field of view of thedriver, then in the cognitive function of the driver, filteringsometimes acts such that, although the information enters as light intothe eyes, the contents of the display information is consciouslyexcluded. Although the filtering effect of normally displayedinformation by the driver becomes one of factors of overlooking ofinformation, the factor can be reduced or avoided by introduction of aninteractive confirmation response procedure with the driver.

Advantageous Effects of Invention

According to the present technology, detailed and timely information ofa section in which driving intervention for a fixed period of time in aplanned traveling route is required can be provided appropriately to thedriver. Further, as a significant effect although it is indirect,driving intervention return of the driver is implemented precisely witha high probability in regard to a vehicle that travels in the automaticdriving mode in a section in which automatic driving is available,another section in which manual driving is requested or in an automaticdriving passable section with care. As a result, also an effect can beanticipated that automatic driving vehicle can be introduced withoutdisturbing normal utilization of the road infrastructure under aninfrastructure environment into which an environment in which automaticdriving is possible is introduced in a mixed manner. It is to be notedthat the advantageous effects described in the present specification areillustrative only and are not restrictive, and other advantageouseffects may be applicable.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram depicting an example of a configuration of avehicle controlling system.

FIG. 2 is a view depicting an example of installation of various sensorsfor detecting external information of an own vehicle.

FIG. 3 is a view schematically depicting an example of a manual takeoversequence of automatic driving by an automatic driving controlling unit.

FIG. 4 is a view depicting a more detailed example of the manualtakeover sequence of automatic driving.

FIG. 5 is a flow chart depicting an overview of operation of anautomatic driving target vehicle including the vehicle controllingsystem.

FIG. 6 is a view depicting an example of a traveling route in whichautomatic driving availability sections determined by setting of adestination by a driver are set in a mixed state or appear.

FIG. 7 depicts views each illustrating information processing for atraveling section display image along the traveling route.

FIG. 8 depicts views each illustrating an example of a traveling sectiondisplay image that is displayed finally.

FIG. 9 depicts views each illustrating an example of change (example ofscroll) of a traveling section display image as time passes.

FIG. 10 depicts views each illustrating an example of a travelingsection display image along a traveling route displayed on a screen of atablet terminal apparatus (hereinafter referred to merely as a“tablet”).

FIG. 11 is a view depicting an example of a state in which a driver isexecuting a secondary task actually using a tablet.

FIG. 12 is a view depicting an example in which a cautious travelingsection Sd appears newly in a second section and a warning of this isgiven to a driver in a flickering display.

FIG. 13 is a view depicting a state in which a small window is pop-updisplayed on a screen of a tablet.

FIG. 14 depicts views each illustrating an example of informationdisplayed in the small window.

FIG. 15 is a view depicting an emphatic display (wave display) when adriver intervention requiring section comes in a range of a fixed periodof time from the current spot in a state in which a traveling sectiondisplay image is displayed on the screen of the tablet.

FIG. 16 is a view depicting an example of display that performs wavedisplay.

FIG. 17 is a view depicting another example of display that performswave display.

FIG. 18 is a view depicting an example in which a milestone point and soforth are further displayed in a state in which a traveling sectiondisplay image is displayed on the screen of the tablet.

FIG. 19 depicts views each illustrating an example in which a workscreen of a secondary task and a map screen can be switchably displayedfurther in response to a selection operation of the driver (user) in astate in which a traveling section display image is displayed on thescreen of the tablet.

FIG. 20 depicts views each illustrating an example of a takeovernotification or a restart return point designation slider menu displayedon a terminal.

FIG. 21 depicts views each illustrating an example of a display ofreduction in a secondary task execution window or the like of a restartreturn point designation slider menu or a secondary task executionwindow in a case where there is no acknowledge response of a performerof a secondary task, which are displayed on a terminal.

FIG. 22 is a view depicting an example of a display of a slider menuthat designates a work restart point displayed on a terminal.

FIG. 23 is a flow chart (1/2) depicting an example of a processingprocedure of the system in a case where a takeover notification decisionis received.

FIG. 24 is a flow chart (1/2) depicting the example of the processingprocedure of the system in the case where the takeover notificationdecision is received.

DESCRIPTION OF EMBODIMENT

In the following, a mode for carrying out the invention (hereinafterreferred to as an “embodiment”) is described. It is to be noted that thedescription is given in the following order.

1. Embodiment

2. Modifications

1. Embodiment

[Configuration of Automatic Driving Controlling System]

FIG. 1 depicts an example of a configuration of a vehicle controllingsystem 100 as the embodiment. It is to be noted that, in a case where avehicle in which the vehicle controlling system 100 is provided isdistinguished from any other vehicle, it is referred to as an own car oran own vehicle.

The vehicle controlling system 100 includes an inputting unit 101, adata acquisition unit 102, a communication unit 103, an in-vehicleapparatus 104, an output controlling unit 105, an outputting unit 106, adrive-train system controlling unit 107, a drive-train system 108, abody controlling unit 109, a body system 110, a storage unit 111, and anautomatic driving controlling unit 112.

The inputting unit 101, the data acquisition unit 102, the communicationunit 103, the output controlling unit 105, the drive-train systemcontrolling unit 107, the body controlling unit 109, the storage unit111, and the automatic driving controlling unit 112 are connected toeach other by a communication network 121. The communication network 121includes an in-vehicle communication network, a bus, or the like thatcomplies with an arbitrary standard such as, for example, CAN(Controller Area Network), LIN (Local Interconnect Network), LAN (LocalArea Network), or FlexRay (registered trademark). It is to be noted thatthe components of the vehicle controlling system 100 are sometimesconnected directly to each other without the intervention of thecommunication network 121.

It is to be noted that, in the description hereinafter given, in a casewhere the components of the vehicle controlling system 100 communicatewith each other through the communication network 121, description ofthe communication network 121 is omitted. For example, in a case wherethe inputting unit 101 and the automatic driving controlling unit 112communicate with each other through the communication network 121, thisis simply described as follows: the inputting unit 101 and the automaticdriving controlling unit 112 communicate with each other.

The inputting unit 101 includes a device that is used to input variousdata, instructions, and so forth by a passenger. For example, theinputting unit 101 includes operation devices such as a touch panel,buttons, a microphone, switches, and levers, as well as operationdevices capable of inputting by a method other than a manual drivingthrough a voice, a gesture, or the like. Further, for example, theinputting unit 101 may be a remote control apparatus that utilizesinfrared rays or other electromagnetic waves or an external connectionapparatus such as a mobile apparatus, a wearable apparatus, or the like,which are ready for an operation of the vehicle controlling system 100.The inputting unit 101 generates an input signal on the basis of data,an instruction, or the like inputted by a passenger and supplies theinput signal to the components of the vehicle controlling system 100.

The data acquisition unit 102 includes various sensors for acquiringdata to be used for processing of the vehicle controlling system 100 andsupplies the acquired data to the components of the vehicle controllingsystem 100.

For example, the data acquisition unit 102 includes various sensors fordetecting a state and so forth of the own vehicle. In particular, forexample, the data acquisition unit 102 includes a gyro sensor, anacceleration sensor, an inertial measurement device (IMU), sensors fordetecting an operation amount of an accelerator pedal, an operationamount of a brake pedal, a steering angle of a steering wheel, an enginespeed, a motor speed, a rotational speed of wheels and so forth, andother necessary sensors.

Further, the data acquisition unit 102 includes various sensors fordetecting information outside the own vehicle, for example. Inparticular, the data acquisition unit 102 includes an imaging apparatussuch as a ToF (Time Of Flight) camera, a stereo camera, a monocularcamera, an infrared camera, and other cameras, for example. Further, thedata acquisition unit 102 includes an environment sensor for detectingthe weather, meteorological phenomenon, or the like, and surroundinginformation detection sensors for detecting an object around the ownvehicle, for example. The environment sensor includes a rain dropsensor, a fog sensor, a sunshine sensor, and a snow sensor, and thelike, for example. The surrounding information detection sensorincludes, for example, an ultrasonic sensor, a radar, a LiDAR (LightDetection and Ranging, Laser Imaging Detection and Ranging), a sonar,and so forth.

For example, FIG. 2 depicts an example of installation of varioussensors for detecting external information of the own vehicle. Imagingapparatuses 7910, 7912, 7914, 7916, and 7918 are provided at at leastone of positions on a front nose, side mirrors, a rear bumper, a backdoor of the vehicle 7900, or at a position on an upper portion of awindshield within the interior of the vehicle.

The imaging apparatus 7910 provided on the front nose and the imagingapparatus 7918 provided at the upper portion of the windshield in theinterior of the vehicle acquire images principally ahead of the vehicle7900. The imaging apparatuses 7912 and 7914 provided on the side mirrorsacquire images principally of the sides of the vehicle 7900. The imagingapparatus 7916 provided on the rear bumper or the back door acquires animage principally behind the vehicle 7900. The imaging apparatus 7918provided at the upper portion of the windshield in the interior of thevehicle is used for detection principally of a preceding vehicle or of apedestrian, an obstacle, a traffic light, a traffic sign, a lane track,and so forth. Further, in automatic driving in the future, the imagingapparatus 7918 may be extensionally utilized to a pedestrian crossing aroad ahead of a left or right turn in an wider area range when thevehicle turns to the left or right or further to a range of anapproaching substance on a crossing road.

It is to be noted that, in FIG. 2, an example of imaging ranges of theimaging apparatuses 7910, 7912, 7914, and 7916 is depicted. An imagingrange a indicates an imaging range of the imaging apparatus 7910provided on the front nose; imaging ranges b and c depict imaging rangesof the imaging apparatuses 7912 and 7914 provided on the side mirrors,respectively; and an imaging range d indicates an imaging range of theimaging apparatus 7916 provided on the rear bumper or the back door. Forexample, by overlaying image data captured by the imaging apparatuses7910, 7912, 7914, and 7916, a bird's eye image of the vehicle 7900viewed from above, an omnidirectional three-dimensional display imagesurrounding the vehicle periphery with curved planes, and so forth areobtained.

The sensors 7920, 7922, 7924, 7926, 2928, and 7930 provided on thefront, rear, sides, and corners of the vehicle 7900 and at the upperportion of the windshield in the interior of the vehicle may be, forexample, ultrasonic sensors or radars. The sensors 7920, 7926, and 7930provided on the front nose, rear bumper and backdoor and at the upperportion of the windshield in the interior of the vehicle may each be,for example, a LiDAR. The sensors 7920 to 7930 are used for detectionprincipally of a preceding vehicle, a pedestrian, an obstacle, or thelike. A result of such detection may be applied further to improvementin three-dimensional object display of the bird's eye display oromnidirectional stereoscopic display.

Referring back to FIG. 1, for example, the data acquisition unit 102includes various sensors for detecting the current position of the ownvehicle. In particular, the data acquisition unit 102 includes, forexample, a GNSS receiver for receiving GNSS signals from GNSS (GlobalNavigation Satellite System) satellites, and so forth.

Further, the data acquisition unit 102 includes various sensors, forexample, for detecting information regarding the inside of the vehicle.In particular, for example, the data acquisition unit 102 includes animaging apparatus for imaging the driver, a biological sensor fordetecting biological information of the driver, a microphone forcollecting sound in the interior of the vehicle and so forth. Thebiological sensor is provided, for example, on a seat face, the steeringwheel, or the like and detects a sitting state of a passenger sitting ona seat or biological information of the driver who grabs the steeringwheel. As a biological signal, diversified observable data of a heartrate, a pulse rate, a blood flow, breathing, a mind-body correlation, avisual stimulus, brain waves, sweating, a drift, a head posturebehavior, the eyes, gaze, a blink, a saccade, a micro saccade, fixation,staring, an iris pupil reaction, and so forth can be utilized.

The communication unit 103 performs communication with the in-vehicleapparatus 104 as well as various apparatuses, servers, base stations,and so forth outside the vehicle to transmit data supplied from thecomponents of the vehicle controlling system 100 and supplies receiveddata to the components of the vehicle controlling system 100. It is tobe noted that the communication protocol supported by the communicationunit 103 is not specifically restricted, and also it is possible for thecommunication unit 103 to support a plurality of kinds of communicationprotocols.

For example, the communication unit 103 performs wireless communicationwith the in-vehicle apparatus 104 through a wireless LAN, Bluetooth(registered trademark), NFC (Near Field Communication), WUSB (WirelessUSB), or the like. Further, the communication unit 103 performs wiredcommunication with the in-vehicle apparatus 104 through a connectionterminal not depicted (and a cable if necessary) by an USB (UniversalSerial Bus), an HDMI (High-Definition Multimedia Interface), an MHL(Mobile High-definition Link), or the like.

Furthermore, the communication unit 103 performs communication, forexample, with an apparatus (for example, an application server or acontrol server) existing in an external network (for example, theInternet, a cloud network or a network unique to a provider) through abase station or an access point. Further, the communication unit 103performs communication with a terminal existing in the neighborhood ofthe own vehicle (for example, a terminal of a pedestrian or a shop or anMTC (Machine Type Communication) terminal), for example, using the P2P(Peer To Peer) technology. Furthermore, the communication unit 103performs V2X communication such as, for example, vehicle to vehicle(Vehicle to Vehicle) communication, road to vehicle (Vehicle toInfrastructure) communication, communication between the own vehicle anda home (Vehicle to Home), and pedestrian to vehicle (Vehicle toPedestrian) communication. Further, the communication unit 103 includesa beacon reception unit, for example, and receives a radio wave or anelectromagnetic wave originated from a wireless station or the likeinstalled on a road to acquire information regarding the currentposition, traffic jams, traffic rules, required time, or the like. It isto be noted that pairing with a vehicle traveling ahead during travelingin a section, the vehicle which can become a leading vehicle through thecommunication unit such that information acquired by a data acquisitionunit incorporated in the vehicle traveling ahead is acquired aspre-traveling information and is used complementarily with the data ofthe data acquisition unit 102 of the own vehicle, and this becomes meansfor assuring higher safety of a line of subsequent vehicles, especiallyin a case of a line traveling by a leading vehicle, followed by thesubsequent vehicles, for example.

The in-vehicle apparatus 104 includes, for example, mobile equipment (atablet, a smartphone, or the like) or a wearable device owned by apassenger, information equipment carried in or attached to the ownvehicle, a navigation apparatus for performing a route search to anarbitrary destination, and so forth. It is to be noted that, if it istaken into consideration that, as a result of spread of automaticdriving, an occupant is not necessarily fixed to a sitting fixedposition, the in-vehicle apparatus 104 may be extensionally utilized toa video reproduction device, a game device, or an apparatus that can beremoved from an installation position of the same. Although the presentembodiment is described in connection with an example in whichpresentation of information of an intervention requiring point of thedriver is restricted to the applicable driver, information provision maybe performed further to a subsequent vehicle in a line traveling or thelike or may be further utilized suitably in combination with travelingsupport at a remote place by normally giving information to a serviceoperation control center for passenger transport carpool buses orlong-distance logistics commercial vehicles.

The output controlling unit 105 controls outputting of various kinds ofinformation to a passenger of the own vehicle or to the outside of thevehicle. For example, the output controlling unit 105 generates anoutput signal including at least one of visual information (for example,image data) and auditory information (for example, sound data) andsupplies the output signal to the outputting unit 106 to controloutputting of the visual information and the auditory information fromthe outputting unit 106. In particular, the output controlling unit 105synthesizes image data captured by a different imaging apparatus of thedata acquisition unit 102 to generate a bird's eye image, a panoramaimage, or the like and supplies an output signal including the generatedimage to the outputting unit 106. Further, the output controlling unit105 generates sound data including, for example, warning sound, awarning message, or the like against a risk of collision, contact,entering into a danger zone, or the like and supplies an output signalincluding the generated sound data to the outputting unit 106.

The outputting unit 106 includes an apparatus capable of outputtingvisual information or auditory information to a passenger of the ownvehicle or to the outside of the vehicle. For example, the outputtingunit 106 includes a display apparatus, an instrument panel, an audiospeaker, a headphone, a wearable device such as a glasses type displayto be worn by a passenger, a projector, a lamp, and so forth. Thedisplay apparatus provided in the outputting unit 106 may be not only anapparatus having an ordinary display, but also an apparatus fordisplaying visual information in the visual field of the driver such as,for example, a head-up display, a transmission type display or anapparatus having an AR (Augmented Reality) display function.

The drive-train system controlling unit 107 generates various controlsignals and supplies them to the drive-train system 108 to performcontrol of the drive-train system 108. Further, the drive-train systemcontrolling unit 107 supplies control signals to the components otherthan the drive-train system 108 to perform notification of a controlstate of the drive-train system 108 and so forth as occasion demands.

The drive-train system 108 includes various apparatuses relating to thedrive system of the own vehicle. For example, the drive-train system 108includes a driving force generation apparatus for generating drivingforce such as an internal combustion engine or a drive motor, a drivingforce transmission mechanism for transmitting the driving force to theaxles, a steering mechanism for adjusting the steering angle, a brakesystem for generating braking force, an ABS (Antilock Brake System), anESC (Electronic Stability Control), an electric power steeringapparatus, and so forth.

The body controlling unit 109 generates various control signals andsupplies them to the body system 110 to perform control of the bodysystem 110. Further, the body controlling unit 109 supplies controlsignals to the components other than the body system 110 to performnotification of a control state of the body system 110, and so forth asoccasion demands.

The body system 110 includes various apparatuses of the body systemequipped on the vehicle body. For example, the body system 110 includesa keyless entry system, a smart key system, a power window apparatus,power seats, a steering wheel, an air conditioning apparatus, variouslamps (for example, a headlamp, a back lamp, a brake lamp, a turnsignal, a fog lamp, and so forth), and so forth.

The storage unit 111 includes magnetic storage devices such as, forexample, a ROM (Read Only Memory), a RAM (Random Access Memory), and anHDD (Hard Disc Drive), a semiconductor storage device, an opticalstorage device, a magneto-optical storage device, and so forth. Thestorage unit 111 stores various programs, data, and so forth to be usedby the components of the vehicle controlling system 100. For example,the storage unit 111 stores map data of a three-dimensional highprecision map such as a dynamic map, a global map that is lower inaccuracy than the high precision map but covers a wider area, a localmap including information around the own vehicle and so forth.

The automatic driving controlling unit 112 performs control relating toautomatic driving such as autonomous traveling, driving assistance andso forth. In particular, for example, the automatic driving controllingunit 112 performs cooperative control for an aim of implementation ofthe ADAS (Advanced Driver Assistance System) functions includingcollision avoidance or impact mitigation of the own vehicle, followtraveling based on the inter-vehicle distance, vehicle speed maintainingtraveling, collision warning of the own vehicle, lane departuretraveling, and so forth. Also, for example, the automatic drivingcontrolling unit 112 performs cooperative control for an aim ofautomatic traveling of autonomously driving without depending onoperation of the driver, and so forth. The automatic driving controllingunit 112 includes a detection unit 131, a self-position estimation unit132, a situation analysis unit 133, a planning unit 134, and a motioncontrolling unit 135.

The detection unit 131 performs detection of various types ofinformation necessary for control of automatic driving. The detectionunit 131 includes an outside-vehicle information detection unit 141, anin-vehicle information detection unit 142, and a vehicle state detectionunit 143.

The outside-vehicle information detection unit 141 performs a detectionprocess of information of the outside of the own vehicle on the basis ofdata or signals from the components of the vehicle controlling system100. For example, the outside-vehicle information detection unit 141performs a detection process, a recognition process, and a trackingprocess of an object around the own vehicle, and a detection process ofthe distance to the object. The objects that become a detection targetinclude, for example, a vehicle, a person, an obstacle, a structure, aroad, a traffic light, a traffic sign, a road sign, and so forth.

Further, the outside-vehicle information detection unit 141 performs,for example, a detection process of an environment around the ownvehicle. Surrounding environments that become a detection targetinclude, for example, the weather, air temperature, humidity,brightness, a state of the road, and so forth. The outside-vehicleinformation detection unit 141 supplies data indicative of a result ofthe detection process to the self-position estimation unit 132, a mapanalysis unit 151, a traffic rule recognition unit 152 and a situationrecognition unit 153 of the situation analysis unit 133, an emergencyavoidance unit 171 of the motion controlling unit 135, and so forth.

As the information to be acquired by the outside-vehicle informationdetection unit 141, information principally by the infrastructure can bereceived if the traveling section is a section in which the localdynamic map that is normally updated mainly as a section in whichtraveling by automatic driving is possible is supplied by theinfrastructure, or the own vehicle may travel normally receiving updateof the map from a vehicle or a vehicle group traveling in thecorresponding section preceding to the own vehicle in advance before theown vehicle advances into a section. Further, in such a case that updatewith the latest local dynamic map from the infrastructure is notperformed normally or in a like case, road environment informationobtained from a leading vehicle which has already entered the sectionmay be further utilized complementarily in order to obtain roadinformation immediately before entering the section more safelyespecially by a line traveling or the like. Whether automatic driving ispossible in the section depends upon whether or not advance informationprovided from the infrastructure is available. Provision of automaticdriving traveling availability information on a route provided from theinfrastructure is equivalent to provision of a so-called invisible trackas information.

The in-vehicle information detection unit 142 performs a detectionprocess of in-vehicle information on the basis of data or signals fromthe components of the vehicle controlling system 100. For example, thein-vehicle information detection unit 142 performs an authenticationprocess and a recognition process of the driver, a detection process ofa state of the driver, a detection process of a passenger, a detectionprocess of an environment in the inside of the vehicle and so forth. Thestates of the driver that become a detection target include, forexample, a physical condition, awakening, concentration, fatigue, aline-of-sight direction, and so forth.

Further, utilization of automatic driving in which the driver iswithdrawn fully from a driving steering work is supposed in the future,and it becomes necessary for the system to grasp that the driver has hada temporary doze or has started some other work and how far theconsciousness necessary for return to driving has returned. In short,although a driver monitoring system investigated heretofore has beendirected principally to means for detecting lower consciousness such assleepiness, since a state in which the driver does not at all intervenewith driving steering is applicable in the future, the system becomesfree from the means for observing the driving intervention degree of thedriver, and it is necessary to observe the consciousness returntransition necessary for driving from a state in which an accurate stateof consciousness of the driver is unknown and, after accurate internalawakening of the driver is grasped, to proceed with transfer ofintervention from automatic driving to manual driving.

Therefore, the in-vehicle information detection unit 142 has mainly twomajor roles, the first one of which is passive monitoring of the stateof the driver during automatic driving and the second one of which is,after a request for return is issued from the system, to detect whetherthe ability of the driver such as peripheral recognition, perception,and determination as well as an operational capability of the steeringequipment reaches a level at which manual driving is possible until asection for driving with caution is reached. As the control, failureself-diagnosis of the entire vehicle may be further performed such that,also in a case where degradation of the automatic driving functionoccurs due to some functional failure for automatic driving, earlyreturn to manual driving by the driver is encouraged similarly. Thepassive monitoring here signifies detection means of the type that doesnot require a conscious response reaction of the driver and does notexclude an article that originates a physical radio wave, light or thelike from an apparatus and detects a response signal. In short, that arecognition response reaction is not under consciousness such as duringa nap is referred to as a passive type.

The environment in the inside of the vehicle that is a detection targetincludes, for example, the temperature, humidity, brightness, smell, andso forth. The in-vehicle information detection unit 142 supplies dataindicative of a result of the detection process to the situationrecognition unit 153 of the situation analysis unit 133 and to themotion controlling unit 135. It is to be noted that, in a case where itis found that, after a driving return instruction to the driver by thesystem is issued, the driver cannot achieve manual driving in precisedeadline time and, even if deceleration control is performed whileself-driving is maintained to perform time-giving, it is determined thattakeover cannot be performed in time, the in-vehicle informationdetection unit 142 issues an instruction to the emergency avoidance unit171 or the like of the system to start a deceleration, escape andstopping procedure in order to escape the vehicle.

The vehicle state detection unit 143 performs a detection process of astate of the own vehicle on the basis of data or signals from thecomponents of the vehicle controlling system 100. The states of the ownvehicle that become a detection target include, for example, a speed, anacceleration, a steering angle, presence or absence and the details ofabnormality, a state of a driving operation, a position and aninclination of the power seat, a locked state of a door, states of otherin-vehicle apparatuses, and so forth. The vehicle state detection unit143 supplies data indicative of a result of the detection process to thesituation recognition unit 153 of the situation analysis unit 133,emergency avoidance unit 171 of the motion controlling unit 135, and soforth.

The self-position estimation unit 132 performs an estimation process ofthe position, posture, and so forth of the own vehicle on the basis ofdata or signals from the components of the vehicle controlling system100, such as the outside-vehicle information detection unit 141, and thesituation recognition unit 153 of the situation analysis unit 133.Further, the self-position estimation unit 132 generates a local map foruse for estimation of the position of the own vehicle (hereinafterreferred to as an own position estimation map) if necessary.

The own position estimation map is a high accuracy map using, forexample, a technology of SLAM (Simulated Localization and Mapping). Theself-position estimation unit 132 supplies data indicative of a resultof the estimation process to the map analysis unit 151, the traffic rulerecognition unit 152, the situation recognition unit 153, and so forthof the situation analysis unit 133. Further, the self-positionestimation unit 132 stores the own position estimation map into thestorage unit 111.

The situation analysis unit 133 performs an analysis process of the ownvehicle and a surrounding situation. The situation analysis unit 133includes the map analysis unit 151, the traffic rule recognition unit152, the situation recognition unit 153, a situation prediction unit 154and a learning unit 155.

The map analysis unit 151 performs an analysis process of various mapsstored in the storage unit 111 while using data or signals from thecomponents of the vehicle controlling system 100, such as theself-position estimation unit 132 and the outside-vehicle informationdetection unit 141 as occasion demands to construct a map includinginformation necessary for processing of automatic driving. The mapanalysis unit 151 supplies the constructed map to the traffic rulerecognition unit 152, the situation recognition unit 153, the situationprediction unit 154 and a route planning unit 161, an action planningunit 162 and a motion planning unit 163 of the planning unit 134.

The traffic rule recognition unit 152 performs a recognition process oftraffic rules around the own vehicle on the basis of data or signalsfrom the components of the vehicle controlling system 100, such as theself-position estimation unit 132, the outside-vehicle informationdetection unit 141, and the map analysis unit 151. By this recognitionprocess, for example, the position and the state of a traffic lightaround the own vehicle, the details of a traffic regulation around theown vehicle, available lanes, and so forth are recognized. The trafficrule recognition unit 152 supplies data indicative of a result of therecognition process to the situation prediction unit 154 and so forth.

The situation recognition unit 153 performs a recognition process of asituation relating to the own vehicle on the basis of data or signalsfrom the components of the vehicle controlling system 100, such as theself-position estimation unit 132, the outside-vehicle informationdetection unit 141, the in-vehicle information detection unit 142, andthe vehicle state detection unit 143, and the map analysis unit 151. Forexample, the situation recognition unit 153 performs a recognitionprocess of the situation of the own vehicle, situation around the ownvehicle, situation of the driver of the own vehicle, and so forth.Further, the situation recognition unit 153 generates a local map to beused for recognition of the situation around the own vehicle(hereinafter referred to as a map for situation recognition) ifnecessary. The map for situation recognition is set as, for example, anoccupancy grid map (Occupancy Grid Map).

The situations of the own vehicle that become a recognition targetinclude, for example, conditions unique to the vehicle such as theposition, posture and movement (for example, the speed, acceleration,moving direction and so forth) of the own vehicle, and the movement ofthe center of gravity of the vehicle body in association with a loadedcargo quantity that determines the motion characteristics of the ownvehicle and loading of the cargo, a tire pressure, braking distancemovement in association with brake braking pad wear situation,permissible maximum deceleration braking for prevention of movement ofthe cargo caused by braking due to loaded cargo, and a centrifugalrelaxation limit speed arising from a liquid cargo at the time oftraveling on a curve, and conditions unique to the loaded cargo, andsince the timing required for control differs depending on theproperties of the vehicle itself in a quite same road environment suchas a coefficient of friction of a road surface, a road curve, or agradient, and on a loaded article, and so forth, it is necessary toperform collection of such various conditions and learn them so as to bereflected upon an optimum timing at which the control is to beperformed. It is not sufficient if the presence or absence and thedetails of abnormality and so forth of the own vehicle are simplyobserved and monitored.

The situations around the own vehicle that become a recognition targetinclude, for example, the type and the position of a stationary objectaround the own vehicle, the type, position and movement (for example,the speed, acceleration, moving direction, and so forth) of a movingobject therearound, the configuration of the road therearound and thestate of the road surface as well as the weather, humidity, brightness,and so forth therearound. The states of the driver that become arecognition target include, for example, the physical condition,awakening level, concentration, fatigue, movement of the line of sight,driving operations, and so forth. For safe traveling of the vehicle, thecontrol start point at which coping is demanded differs significantlydepending upon a loaded amount loaded in the unique state of thevehicle, a chassis fixed state of the loading portion, a biased state ofthe center of gravity, a maximum possible-deceleration accelerationvalue, a maximum load possible centrifugal force, and a return responsedelay mount in response to the state of the driver.

The situation recognition unit 153 supplies data indicative of a resultof the recognition process (including, as occasion demands, the map forsituation recognition) to the self-position estimation unit 132, thesituation prediction unit 154, and so forth. Further, the situationrecognition unit 153 stores the map for situation recognition into thestorage unit 111.

The situation prediction unit 154 performs a prediction process of asituation relating to the own vehicle on the basis of data or signalsfrom the components of the vehicle controlling system 100 such as themap analysis unit 151, the traffic rule recognition unit 152, thesituation recognition unit 153, and so forth. For example, the situationprediction unit 154 performs a prediction process of a situation of theown vehicle, a situation around the own vehicle, a situation of thedriver and so forth.

The situations of the own vehicle that become a prediction targetinclude, for example, a behavior of the own vehicle, occurrence ofabnormality, a mileage, and so forth. The situations around the ownvehicle that become a prediction target include, for example, a behaviorof a moving object around the own vehicle, a change of the state of atraffic light, a change of the environment such as the weather, and soforth. The situations of the driver that become a prediction targetinclude, for example, a behavior, a physical condition, and so forth ofthe driver.

The situation prediction unit 154 supplies data indicative of a resultof the prediction process to the route planning unit 161, the actionplanning unit 162, the motion planning unit 163, and so forth of theplanning unit 134 together with data from the traffic rule recognitionunit 152 and the situation recognition unit 153.

The learning unit 155 learns an optimum return timing according to areturn action pattern of the driver, a vehicle property and so forth andsupplies learning information to the situation recognition unit 153 andso forth. Consequently, it is possible, for example, to present to thedriver an optimum timing that is required for the driver to return fromautomatic driving to manual driving at a prescribed fixed ratio or moreand is determined statistically.

The route planning unit 161 plans a route to a destination on the basisof data or signals from the components of the vehicle controlling system100 such as the map analysis unit 151 and the situation prediction unit154. For example, the route planning unit 161 sets a route from thecurrent position to a designated destination on the basis of the globalmap. Further, for example, the route planning unit 161 suitably changesthe route on the basis of a situation such as a traffic jam, anaccident, a traffic rule, or a construction, the physical condition ofthe driver, and so forth. The route planning unit 161 supplies dataindicative of the planned route to the action planning unit 162 and soforth.

The action planning unit 162 plans an action of the own vehicle fortraveling safely in a planned period of time along the route planned bythe route planning unit 161 on the basis of data or signals from thecomponents of the vehicle controlling system 100 such as the mapanalysis unit 151 and the situation prediction unit 154. For example,the action planning unit 162 performs planning of start, stop, a movingdirection (for example, forward, backward, turn left, turn right,turnaround, or the like), a traveling lane, a traveling speed, passing,and so forth. The action planning unit 162 supplies data indicative ofthe planned action of the own vehicle to the motion planning unit 163and so forth.

The motion planning unit 163 plans a motion of the own vehicle forimplementing the action planned by the action planning unit 162 on thebasis of data or signals from the components of the vehicle controllingsystem 100 such as the map analysis unit 151 and the situationprediction unit 154. For example, the motion planning unit 163 performsplanning of acceleration, deceleration, a traveling track, and so forth.The motion planning unit 163 supplies data indicative of the plannedmotion of the own vehicle to an acceleration/deceleration controllingunit 172, a direction controlling unit 173, and so forth of the motioncontrolling unit 135.

The motion controlling unit 135 performs control of motion of the ownvehicle. The motion controlling unit 135 includes an emergency avoidanceunit 171, an acceleration/deceleration controlling unit 172, and adirection controlling unit 173.

The emergency avoidance unit 171 performs a detection process of anemergency such as collision, contact, entering of a danger zone,abnormality of the driver or abnormality of the vehicle on the basis ofresults of detection of the outside-vehicle information detection unit141, the in-vehicle information detection unit 142, and the vehiclestate detection unit 143. In a case where the emergency avoidance unit171 detects occurrence of an emergency, it plans a motion of the ownvehicle for avoiding the emergency such as sudden stop, sharp turn, orthe like. The emergency avoidance unit 171 supplies data indicative ofthe planed motion of the own vehicle to the acceleration/decelerationcontrolling unit 172, the direction controlling unit 173, and so forth.

The acceleration/deceleration controlling unit 172 performsacceleration/deceleration control for implementing the motion of the ownvehicle planned by the motion planning unit 163 or the emergencyavoidance unit 171. The acceleration/deceleration controlling unit 172calculates a control target value of the driving force generationapparatus or the brake system for implementing the planned acceleration,deceleration or sudden stop and supplies a control instructionindicative of the calculated control target value to the drive-trainsystem controlling unit 107. It is to be noted that an emergencypossibly occurs principally in two cases. In particular, the two casesinclude a case in which an unexpected accident occurs due to a suddenreason during automatic driving on a road that is originally determinedto be safe in a local dynamic map or the like acquired from theinfrastructure in the traveling route during automatic driving andemergency return is not performed in time and another case in which thedriver cannot return precisely from automatic driving to manual driving.

The direction controlling unit 173 performs direction control forimplementing the motion of the own vehicle planned by the motionplanning unit 163 or the emergency avoidance unit 171. For example, thedirection controlling unit 173 calculates a control target value of thesteering mechanism for implementing a traveling track or sharpturnaround planned by the motion planning unit 163 or the emergencyavoidance unit 171 and supplies a control instruction indicative of thecalculated control target value to the drive-train system controllingunit 107.

[Manual Takeover Sequence of Automatic Driving]

FIG. 3 schematically depicts an example of a manual takeover sequence ofautomatic driving by the automatic driving controlling unit 112. At stepS1, the driver is possibly in a state fully withdrawn from drivingsteering in the future. In this state, the driver can, for example, takea nap, watch a video, focus on a game, or execute a secondary task suchas a work using a visual tool such as a tablet or a smartphone. Also, itcan be considered that the work in which a visual tool such as a tabletor a smartphone is used is performed, for example, in a state in whichthe driver's seat is displaced or on a seat different from the driver'sseat.

Depending on the states of the driver, it is supposed that, when the ownvehicle approaches a section in which manual driving return is demandedon the route, the period of time until the driver returns variessignificantly depending upon the contents of the current work each time,and in a case where the period of time is insufficient with anotification immediately before the approach of the event or thenotification is issued exceedingly early with a margin taken to theapproach of the event, such a situation occurs that the time till atiming at which return is actually required is excessively long. As aresult, if such a situation that a notification is not issued at aprecise timing occurs repeatedly, then the driver will lose thereliability on the timing in regard to the notification timing of thesystem, and the consciousness of the driver in regard to thenotification degrades and the driver will neglect precise measures. Thisresults in increase of the risk that takeover is not performed well andmakes an inhibiting factor of relieved secondary task execution.Therefore, in order for the driver to start measures for precise drivingreturn to the notification, it is necessary for the system to performoptimization of the notification timing.

At step S2, the timing for return notification comes and a notificationfor driving return is issued visually or auditorily to the driver. Theautomatic driving controlling unit 112 monitors, for example, a steadystate of the driver and grasps a timing for issuing a notification, anda notification is issued at a suitable timing. In particular, during apassive monitoring period at the preceding stage, the execution state ofa secondary task by the driver is normally monitored passively and anoptimum timing for the notification can be calculated by the system. Thepassive monitoring during the period at step S1 is normally andcontinuously performed, and the optimum return timing and the returnnotification method are desirably performed in accordance with theunique return characteristic of the driver.

In particular, it is desirable to present, to the driver, an optimumtiming determined statistically and required to allow the driver toreturn from automatic driving to manual driving correctly at a rateequal to or higher than a prescribed rate by learning an optimum returntiming according to a return action timing of the driver, a vehiclecharacteristic, and so forth. In this case, in a case where the driverdoes not respond to the notification within a fixed period of time, awarning by alarming sound is issued.

At step S3, it is checked whether or not the driver returns to be seatedon the driver's seat. At step S4, an internal awakening state of thedriver is checked by a saccade or the like. At step S5, the stability ofan actual steering situation of the driver is monitored. Then at stepS6, takeover from automatic driving to manual driving is completed.

FIG. 4 depicts a more detailed example of the manual takeover sequenceof automatic driving. At step S11, prediction of a return point isvisually presented to a visual tool such as a tablet or a smartphone.However, there is no necessity to restrict the display to the visualtool, but a display mode is desirable in which the visual presentationis included in the field of view of the driver who has been withdrawnfrom driving during execution of a secondary task such as, for example,the center information display of the vehicle. Although details arehereinafter described, presentation of a future plan and approachinformation is performed, and the return point is displayed such that itapproaches the own vehicle as time passes.

At step S12, the presentation contents of the future plan and theapproach information are suitably changed by update of the LDM (LocalDynamic Map) or the like. Further, at step S12, the state of the driveris periodically monitored. At step S13, the return point coming in afixed period of time from the current point is emphatically displayed tothe driver by flickering display, wave display, or the like to notifythe driver. The timing at which the notification is to be issued at stepS13 is adjusted such that return is performed in time by executing thisnotification early in response to a result of the detection periodicallymonitored at the preceding stage, namely, in response to the depth ofwithdrawal of the driver from driving by nap or by a secondary task.

At step S14, if the driver does not react with the notification, then analarm for wakeup is sounded. At step S15, if the driver does not sit onthe driver's seat, then, a visual or auditory notification for return isissued to the driver. At step S16, in a case where there is a returndelay for sitting, an alarm for alert is sounded. At step S17, forexample, a pointing signal for forward confirmation by the driver ismonitored as return start.

At step S18, a sitting posture is checked. At step S19, the internalawakening level of the driver by such a perceptual reflex as circade,fixation, or the like is decided in order to detect recovery of aninternal perception state in the brain of the driver using such means,for example, as detailed analysis of a line of sight. At step S20, thedriving steering authority is sequentially entrusted to the driver, andwhile observing a response steering situation of the actual steeringreaction, the steering is entrusted, and the steering stability ismonitored.

In a case where the result of the monitoring observation of thestability indicates that subjective driving steering return is notdetected to such a degree as is anticipated to the driver, thissignifies that there is a risk that the driver still is in an extendeddreaming state. Therefore, if it is assumed at step S21 that normalreturn is impossible, then it is determined that emergency takeoverresults in failure, and a deceleration crawl escape sequence is started.

Since the temporal transition required for such return changes dependingupon various factors such as an age or experience, fatigue, and so forthof the driver, it is determined by a return timing according to theindividual. In a case where the driver is requested to return fromautomatic driving to manual driving, at least a fixed period of time isrequired until the driver can return to manual driving almost withcertainty. A configuration that performs optimum timing notificationaccording to vehicle characteristics, road characteristics, and adriving return characteristic of an individual is most desirable, andsince an approach situation of a takeover point is displayed in aninterlocking relationship, the driver can obtain the convenience thatthe driver can utilize a secondary task at ease in response to apermission situation and simultaneously, since one of nervous andrelaxed states appears alternately, cautious return of the user isimplemented suitably for every necessary section, ergonomically leadingto utilization. In other words, the driver is released from anunnecessary continuous cautious tension state.

A maximum effect of this procedure resides in that the balance of asecondary task in tension and relaxation states of the user can be kept.As a result, such a risk that, as a result of sustained tension, theuser is tired in nerve and, as a result, the user transits to adistracted state because of drowsiness from the middle can be reduced,and there is an effect in long-term sustainable operation. It is to benoted that, in order to maximize the effect, even in a section in whichdriver intervention is not required for a long distance, in a case wheretension return of the driver is desirable in the middle, a dummy eventmay be generated in the middle such that return level evaluation of thedriver and aptitude response evaluation of the driver are performed inresponse to the return degree of the dummy event by the driver andrecording storage of the return degree evaluation values may beperformed further.

[Overview of Motion of Automatic Driving Target Vehicle]

A flow chart of FIG. 5 depicts an overview of motion of an automaticdriving target vehicle that includes the vehicle controlling system 100described hereinabove. At step S31, motion is started. Then, at stepS32, the inputting unit 101 will be operated by the driver to set adestination. In this case, an inputting operation of the driver isperformed on the basis of a display image on the instrument panel.

It is to be noted that, although an example of a case in which a usergets on the vehicle to suppose itinerary setting is described as thepresent embodiment, the user may perform remote advance reservationsetting from a smartphone in advance before the user gets on the vehicleor from a personal computer before the user goes out of the own home.Further, the system of the vehicle may perform preplanning setting inaccordance with a schedule assumed by the driver in accordance with aschedule table and may update or acquire LDM information regarding aroad environment to display actual traveling advice like a conciergeupon or before getting on the vehicle.

Then at step S33, traveling section display on the traveling route isstarted. This traveling section display is displayed on the instrumentpanel or is displayed on a tablet or the like, on which the driverperforms a secondary task, in a lined up relationship with a workwindow. Consequently, the driver who is performing a work on the workwindow can easily recognize a driver intervention requiring section andan automatic driving available section of the traveling route on a reachprediction time axis from the current point.

In this traveling section display, presentation of the future plan andapproach information to individual points is performed. In thistraveling section display, the driver intervention requiring section andthe automatic driving available section of the traveling route aredisplayed on the reach prediction time axis from the current point.Then, the driver intervention requiring section includes a manualdriving section, a takeover section from automatic driving to manualdriving, and a cautious traveling section from automatic driving.Details of the traveling section display are hereinafter described.

Then at step S34, acquisition of LDM update information is started.Together with the acquisition of LDM update information, it becomespossible to change the contents of the traveling section display to thelatest state. Then at step S35, traveling is started. Then at step ST36,display of the traveling section display is updated on the basis of theposition information of the own vehicle and the acquired LDM updateinformation. Consequently, the traveling section display is scrolldisplayed such that each section approaches the own vehicle as thevehicle travels.

Then at step S37, monitoring of the driver state is performed. Then atstep S38, it is decided whether it is necessary to change the driverstate. For example, in a case where a driver intervention requiringsection approaches in a state in which the own vehicle currently is inan automatic driving available section and the driver is in a fullywithdrawn state from traveling steering, it is necessary to cause thedriver to return to a traveling steering state early.

When there is no necessity to change the driver state, the processingreturns to the operation at step S36. Conversely, when it is necessaryto change the driver state, a notification of this is issued to thedriver at step S39. If the driver does not respond to the notification,then a warning (alarm) is generated. If the driver reacts with thenotification or the warning, then the processing returns to theoperation at step S36.

It is to be noted that, though not represented in the flow chart of FIG.5, for example, in a case where it is estimated, at a stage at which adriver intervention requiring section is approaching, that it isimpossible for the driver to return to a driving steering state in a setgrace scheduled time period, for example, a deceleration crawl escapesequence is started, and arriving time at a takeover point is delayed togenerate time grace. Then, when return cannot be expected even withthis, the vehicle further travels emergently escaping to a shoulder ofthe road or an evacuation spot and is stopped. In regard to whetherarrival at a takeover point is to be delayed by deceleration crawl,since a desirable state is caused by various factors such as aninfrastructure factor such as whether a permissible traffic congestionimpact to or presence or absence of an escape place for the road or atraffic amount at the applicable point of time, dealing according tooptimum control by road-vehicle cooperation is desirable. However, sincethe present specification focuses on a display form, detaileddescription is not given of vehicle control having an influence uponsetting on the time axis.

[Details of Traveling Section Display]

FIG. 6 depicts an example of a traveling route determined throughsetting of a destination by a driver. The traveling route includes anautomatic driving available section Sa, a manual driving section Sb, atakeover section Sc from automatic driving to manual driving and acautious traveling section Sd from automatic driving. Here, the takeoversection Sc exists immediately before the manual driving section Sbwithout fail, and it is necessary for the driver to be in a returnposture to manual driving. Further, the cautious traveling section Sd isa section in which the vehicle can decelerate and travel with automaticdriving maintained under careful watching of the driver who is in areturn posture to manual driving.

In the example depicted, the automatic driving available section Sa isindicated in green; the manual driving section Sb is indicated in red;and the takeover section Sc and the cautious traveling section Sd areindicated in yellow. It is to be noted that, for the convenience ofillustration, the colors are represented in different patterns.

In traveling section display on a display device such as the centerinformation display or a tablet, such sections of the traveling route asdescribed above are represented on the reach prediction time axis fromthe current point. The automatic driving controlling unit 112 performsinformation processing for the traveling section display along thetraveling route on the basis of the traveling route information andtraffic information.

FIG. 7(a) represents individual sections of a traveling route in a fixedscale on the moving distance axis from the current point. FIG. 7(b)represents the flow speed v(t) in average road traffic at individualpoints. FIG. 7(c) represents the sections converted to those on the timeaxis using the speed v(t) from those represented on the moving distanceaxis. Consequently, the sections of the traveling route are presented onthe reach prediction time axis from the current point. In short, aphysical distance on the traveling route can be represented on a timeaxis by dividing the same by an average speed for each section.

In this embodiment, all sections displayed as traveling sections aredivided into three sections as depicted in FIG. 7(d), and the time axisis changed among the sections. In particular, the first section from thecurrent point to a first point (time t0, for example, approximately 10minutes) is displayed as a time linear display nearest section on afirst time axis. For example, time t0 is set to time necessary andsufficient before a general driver ends a secondary task and returns todriving. Since the nearest section approaching by traveling has a visualintuition effect equivalent to that when it is indicated on a map onwhich the vehicle proceeds at a fixed speed, the driver can startprecise preparations for driving return arising from event approach, andthere is a merit that the driver can intuitively and somewhat accuratelygrasp a point at which return is to be started. In short, the displaypurpose of the sections resides in provision of start determinationinformation of a precise return point of a driver to a user.

Meanwhile, the second section from the first point (time t0) to thesecond point (time t1, for example, approximately one hour) is displayedas a time reciprocal display section on a time axis that sequentiallychanges from the first time axis to a second time axis that is reducedat a predetermined ratio from the first time axis. The display purposeof the second section is a scheme for providing a road situation for alonger period accurately to the driver with a narrow display because,when the second section is displayed in a scale factor equal to thatprincipally in the preceding first section, it becomes difficult todisplay the long period in a narrow display space. By the scheme, thedriver can easily grasp up to which point the driver may not berequested to intervene with the driving in a certain fixed section aheadtogether with traveling, and there is a merit that the driver canperform engagement in a secondary task systematically.Necessity/unnecessity of driving intervention becomes clear to thedriver, and in a secondary task involved in communication with a thirdparty or the like, a significant role of information presentation isplayed in release planning of the driver from the secondary task or thelike.

Here, a setting method of this second display section is described withreference to FIG. 7(d). When the height of a triangle is represented byh0, time t at the point earlier by h from the top of the triangle iscalculated by the following expression (1).

t=t0*h0/h  (1)

Meanwhile, the second time axis at the second point (time t1) is reducedat a ratio of hs/h0 from the first time axis. For example, in a casewhere hs=h0/8, the reduction ratio is 1/8.

The display in the second display section described above is, in a casewhere the vehicle is traveling at a fixed vehicle speed, equivalent todisplay where a traveling straight extension display section on the mapis inclined obliquely to the moving direction or to a state in which thefront of the road plane is viewed obliquely. In other words, since thevisual effect of this display section is that the perspective can beunderstood intuitively from the display picture height position, even ifgraduations or the like for accurate position display are not displayedon the screen, the sensory distance can be grasped easily. Then,although a remote section is reduced, since this is not a point at whichthe vehicle arrives quickly by traveling, although rough prediction isimportant, the driver need not intuitively grasp such accurate reachtime information as that at a near point. Therefore, this is preferablealso when the driver schedules secondary task execution.

Further, the third section from the second point (time t1) to the thirdpoint (time t3) is displayed on the second time axis (reduction ratiohs/h0) as a time linear display remote section. By displaying the threesections divided in this manner, the driver can know details of sectioninformation nearest in time with a limited display space and can knowsection information more remote in time. It is to be noted that, when aremote portion is displayed with the display form for the second sectionmaintained, this becomes lower than the human visual resolution andfurther becomes lower than the display resolution limit of the system.Therefore, it becomes impossible to discriminate information necessaryfor plan determination of a secondary task, and the significance of thedisplay function is lost. Therefore, it is the most effective displaythat reduction of the display scale is ended at a stage at which a timesection sense can be sufficiently grasped intuitively and classificationof a necessary intervention section and an unnecessary section isdisplayed appropriately and, in the following sections, display with ascale returned to the fixed scale is performed.

It is to be noted that the vehicle controlling system 100 has defaultvalues for time t0, t1 and t3. Since also it is considerable that thevalues of time t0, t1 and t2 are made different between long distancedriving and short distance driving, the default values are not limitedto one set, and a plurality of sets may be provided such that the driver(user) or the system selectively uses them in response to a travelingroute. Also, it is considerable to allow the driver (user) toarbitrarily set the values of time t0, t1 and t3.

FIGS. 8(a) and 8(b) each depict an example of a traveling sectiondisplay image that is displayed finally. It is to be noted that, by thelength of an arrow mark, whether the time axis is linear and further, achange of the reduction ratio of the time axis are depicted. In the caseof FIG. 8(a), all of the sections of the first, second and thirdsections are displayed with a first width maintained.

On the other hand, in the case of FIG. 8(b), the first section from thecurrent point to the first point (time t0) is displayed with a firstwidth; the second section from the first point (time t0) to the secondpoint (time t1) is displayed with a width sequentially changing from thefirst value to a second value that indicates a width narrower than thefirst width; and the third section from the second point (time T1) tothe third point (time T2) is displayed with the second width.Consequently, the driver can visually recognize the degree of reductionof the time axis in the second and third sections with respect to thefirst section. In short, although the display form in FIG. 7 is adisplay image in which only the reduction ratio in the moving directionis taken into consideration, by further changing the transverse widthwith respect to the moving direction of the display informationartificially in accordance with the perspective, a perspective effectsame as that obtained when the driver views toward the infinitedirection along the progress of a road or the map is obtained, andintuitive grasping of a distribution of driving intervention requiringsections is facilitated in comparison with that when the screen isviewed at a moment. Especially, in a case where only the second sectionis turned in the counterclockwise direction and viewed, since the secondsection is comparable with the road width of the road ahead and thearriving time at each applicable point in a case where the vehicletravels at a fixed speed, even if an accurate position graduation is notdetermined by observation, the reaching feeling to each point can begrasped intuitively, and the display form is considered to allow fortime distribution.

It is to be noted that, when, at a portion at which the reduction ratehs/h0 is low, for example, like the third section, a section of a shorttime length is displayed with the time length as it is, then the sectionis displayed very thin, and it is expected that recognition of thedriver is difficult. Therefore, even in a case where a driverintervention section (manual driving section, takeover section, cautioustraveling section) is actually equal to or shorter than a fixed timelength, it is displayed with a fixed time length. In this case, forexample, in a case where a takeover section and a manual driving sectioncontinue, the display of the takeover section is sometimes omitted. InFIGS. 8(a) and 8(b), the display of the first manual driving section Sbin the third section indicates such a state as just described.Consequently, in the third section whose time axis is reducedsignificantly, the driver intervention requiring section of a short timelength can be displayed such that it can be recognized by the driver.

Further, at a portion at which the reduction rate hs/h0 is low like thethird section, in a case where a manual driving section Sbintermittently appears in a short cycle, this is displayed as a manualdriving section Sb the entirety of which is continuous. In FIGS. 8(a)and 8(b), the display of the second manual driving section Sb in thethird section indicates such a state that it is displayed in such acontinuous manner as just described. The manual driving section Sbdisplayed in this manner actually includes, as depicted in FIG. 8(c), acautious traveling section Sd and an automatic driving available sectionSa of a short period in addition to the manual driving section Sb. It isto be noted that, as hereinafter described, detailed display can beperformed if, in a state in which traveling section display is given ona tablet or the like, the point is, for example, double-touched.

The traveling section display on the traveling route described above issuccessively updated on the basis of position information of the ownvehicle and acquired LDM update information. Consequently, as timepasses, the traveling section display is scroll displayed such that thesections successively approach the own vehicle. FIGS. 9(a) to 9(d) eachdepict an example of change of traveling section display together withpassage of time. Although this example depicts an example in which thesecond section is displayed in a tapering fashion, a case in which allsections are displayed with an equal width is also similar.

In this case, in the first section, the movement in each section ishigh. Meanwhile, in the second section, since the reduction of the timeaxis decreases from the third section side toward the first sectionside, the movement in each section becomes fast. Further, in the thirdsection, since the reduction of the time axis becomes great, themovement in each section is slow.

FIGS. 10(a) and 10(b) each depict an example of a traveling sectiondisplay image 200 on a traveling route displayed on the screen of atablet 300. FIG. 10(a) depicts an example in a case where the tablet 300is used vertically. In this case, the vehicle controlling system 100 isdisplayed in a bent state along the left side and the upper side and isdisplayed in parallel to a work window that is an execution screen for asecondary task performed on the tablet 300. FIG. 10(b) depicts anexample in a case where the tablet 300 is used in landscape. Also inthis case, the traveling section display image 200 is displayed in abent state along the left side and the upper side and is displayed inparallel to a work window that is an execution screen for a secondarytask performed on the tablet 300. It is to be noted that, although, inthe example depicted, the traveling section display image 200 isarranged in a bent state on the screen of the tablet 300, in a casewhere a sufficient arrangement space can be assured, also it can beconsidered that the traveling section display image 200 is arrangedlinearly.

FIG. 11 depicts an example of a state in which the driver executes asecondary task actually using the tablet 300. In this example, thetablet 300 is used in landscape. On the screen of the tablet 300, atraveling section display image 200 is displayed in a state in which itis bent along the left side and the upper side. It is to be noted thatwhether or not the traveling section display image 200 is to bedisplayed on the screen may be selectively performed by an operation ofthe driver (user). In this case, for example, in a case where thetraveling section display image 200 is displayed on the screen, in acase where a driver intervention requiring section comes in a fixedperiod of time and this is to be notified to the driver, then thetraveling section display image 200 may be displayed on the screenautomatically.

In a case where, in a state in which the traveling section display image200 is displayed on the screen of the tablet 300, a driver interventionrequiring section appears newly within the display section, a displayimage of the driver intervention requiring section generated newlyappears newly. In this case, the newly appearing driver interventionrequiring section is displayed, for example, by flickering display for afixed period of time so as to be identifiable from the other displayimage. This flickering display may be accompanied by caution alarmsound. Here, a case where a driver intervention requiring sectionappears newly includes not only a case where a cautious travelingsection or a manual driving section nearly appears but also a case inwhich the traveling section changes from a cautious traveling section toa manual driving section.

FIG. 12 depicts a state in which a cautious traveling section Sd appearsnewly in the second section and this is warned to the driver byflickering display. It is to be noted that, in this case, it may be madepossible to stop the flickering display, namely, the warning state, bythe driver touching the display location of the cautious travelingsection Sd in the flicking display state. As an alternative, it may bemade possible to popup display a small window by the driver touching thedisplay location of the cautious traveling section Sd in the flickingdisplay state and then stop the flickering, namely, the warning state,by the screen touch of the driver for approval.

Further, in a case where, in the state in which the traveling sectiondisplay image 200 is displayed on the screen of the tablet 300, thedriver (user) double-touches to designate an arbitrary point, forexample, a small window is popup displayed, and display associated withthe point is performed as depicted in FIG. 13.

FIGS. 14(a), 14(b), 14(c) and 14(d) each depict an example ofinformation displayed in a small window. FIG. 14(a) depicts an examplein which a road surface situation at the point, weather, what sectionthe current section is and so forth are displayed in text in the smallwindow. FIG. 14(b) depicts an example in which a map at the point isdisplayed in the small window. FIG. 14(c) depicts an example in whichfrom what sections a fixed period of time including the point isconfigured is displayed in the small window. FIG. 14(d) depicts anexample in which a detour is displayed in the small window. It is to benoted that those display examples are illustrative only, and they arenot restrictive. Further, all of them or some of them may besequentially switchably displayed by an operation of the driver.

Further, when a driver intervention requiring section comes into a rangeof a fixed period of time from the current point in a state in which thetraveling section display image 200 is displayed on the screen of thetablet 300, the driver intervention requiring section is put into anemphatically displayed state to issue a notification to the driver forcalling for attention. The driver will quickly perform driving return onthe basis of the notification. It is to be noted that the fixed periodof time for allowing, when the timing comes on the basis of a timemargin necessary for driving return, a notification to be provided as astart signal to the driver is set such that it is sufficiently possiblefor the driver to return to driving before a driving takeover section ora cautious traveling section comes in response to the personality of thedriver or in response to the current state of the driver, or further inresponse to loading braking characteristics of the vehicle and asituation of the road. In short, the detailed signaling point of any ofsuch notifications as described above may be set by learning uniquecharacteristics of driver specification and so forth by a driverindividual property learner and performing the signaling at an optimumtiming. Although the present specification does not specificallydescribe a configuration or a learning method of a learner for personalcharacteristics of the optimum return timing, the learner is learningequipment that estimates, from an observable state of driving, delaytime when the driver returns correctly from automatic driving to manualdriving at a fixed rate after the driver accepts a notification.

In an emphatically displayed state, for example, flickering display,display in a different color, illusion display by which the moving speedlooks higher than an actual speed or, for example, wave display is used.This makes it possible for the driver to easily recognize that a driverintervention requiring section comes in a range of a fixed period oftime from the current point. Although, from a relationship of thedisplay range of the screen or the like, there is a case that thedisplay scale is small in normal approach display and the approachingfeeling is less likely to be grasped, to make use of the most of thedynamic eyesight effect of human eyes, then the changing point of theluminance gradient quick to the approaching location can be generated onthe display screen by this method. Also in a case where the driverperforms a work looking at a different location of the screen on asecondary task, it is the most significant advantage of the presentmethod that the change can be grasped in a peripheral field of view.

FIG. 15 depicts that, as a state of emphatic display, a wave display isapplied to a takeover section Sc existing immediately before a manualdriving section Sb. FIG. 16 depicts an example of a display image inwhich wave display is performed. In this display example, by changingthe section length of the takeover section Sc for each Δt, operationsfor displaying faster than the progress speed and pulling back thedisplay image are repeated. Meanwhile, FIG. 17 depicts another exampleof a display image in which wave display is performed. In this displayexample, by changing the inter-section position of the takeover sectionSc for each Δt, operations for displaying faster than the moving speedand pulling back are repeated.

Further, in a state in which the traveling section display image 200 isdisplayed on the screen of the tablet 300, a planned passage milestonepoint (Milestone Point), a main route name, and so forth may bedisplayed concurrently with the traveling section display image 200.This milestone point is either recommended by the system or set by thedriver (user). FIG. 18 depicts an example of a display image of amilestone point and so forth.

Further, in a state in which the traveling section display image 200 isdisplayed on the screen of the tablet 300, it may be made possible tofurther allow switchable display between a work screen for a secondarytask and a map screen in response to a selection operation of the driver(user). In FIG. 19(a), a work screen for a secondary task is displayedon the screen of the tablet 300. If, in this state, a map screen icon“M” is touched after an arbitrary point of the traveling section displayimage 200 is triple-touched, then a state is entered in which anenlarged map at the point is displayed in a bird's eye view on thescreen of the tablet 300 as depicted in FIG. 19(b).

Further, when the tablet 300 is turned along a horizontal line in thestate in which the enlarged map at the applicable point is displayed onthe screen of the tablet 300 in this manner, the map display enters a 3Ddisplay state as depicted in FIG. 19(c). It is to be noted that, in acase where the arbitrary point is a point very near to the currentpoint, for example, a point in the first section, oblique 3D display isnot performed and such a bird's eye view as depicted in FIG. 19(b) ismaintained. This is because, in the case of the bird's eye display, theapproaching time period changes in proportion to the display length andthe bird's eye display allows the approaching time period to be graspedintuitively. By touching a work screen icon “T” in a state in which theenlarged map is displayed on the screen of the tablet 300, the displayreturns to the state of FIG. 19(a).

As described hereinabove, in the vehicle controlling system 100 depictedin FIG. 1, a driver intervention requiring section and an automaticdriving available section of a traveling route are displayed on a reachprediction time axis from the current point on a display device (aninstrument panel, a tablet, or the like) on the basis of traveling routeinformation and traffic information. Therefore, section information ofthe traveling route can always be provided to the driver appropriately,and it becomes possible for the driver to perform secondary taskexecution and intervention return with clear distinction smoothly andseamlessly without having (return) waiting stress even in a complextraveling level section in which a plurality of automatic drivingtraveling levels are mixed complicatedly. Thus, accomplishment oftraveling in a long section that makes the most of the advantages of theautomatic driving mode becomes possible. Then, generally speaking, sincethe ratio of vehicles that fail in takeover from automatic driving tomanual driving can be reduced effectively, against a significant problemof traffic bandwidth burden of roads as the social infrastructure, it ispossible to minimize the negative influence thereof by introducingautomatic driving widely.

2. Modifications

[Modification 1]

It is to be noted that, in the present specification, the term “system”is used to represent an aggregation of a plurality of components(apparatuses, modules (parts), and so forth) and it does not matterwhether or not all components are accommodated in the same housing.Accordingly, both of a plurality of apparatuses accommodated in separatehousings and connected to each other through a network and one apparatuswhere a plurality of modules are accommodated in a single housing are asystem.

[Modification 2]

Further, in the present specification, in a case where a secondary taskexecuted by a driver is, for example, a video conference system thatincludes a third party, the third party remotely connected to the driverduring the secondary task cannot directly grasp a situation in regard towhen it is necessary for the driver to return to driving. However,since, in this instance, it is unknown when it becomes necessary for thedriver to exit from the secondary task and return to driving, theremotely connected third party cannot grasp and cannot know whether asituation for interruption of communication is imminent, which isinconvenient. Therefore, by providing return necessity timinginformation of a driver to a third party of a connection partner locatedon the opposite side of the connection screen of the video conference inwhich the driver is engaged, it is made well-known that the driverperforms conference execution as a secondary task of a driving work. Inshort, also it is desirable for the connection partner to grasp thesituation of a return timing of the driver, and driver interventionreturn request information over part of time or a fixed period of timeto the partner side may be shared by communication.

Further, in order to make it possible to preferentially work on adriving return request by the system even during a conference, suchmeans as notification to the connection partner side, situationrecognition, conference continuation refusal notification may be taken.As the return request information by the system, information same asthat to the driver need not necessarily be presented, and notificationby voice or message display by an OSD may be used. Furthermore, in acase where voluntary driving intervention return by the driver isdelayed by execution of a secondary task, the system may proceed withinterruption compulsorily. Further, in order to avoid unreasonablecontinuation of a secondary task by the driver, a log of a return actionof the driver from a return request by the system may be trackingrecorded. In short, in a case where a third party remotely connects toand joins in the work of the vehicle driver during automatic driving, itis not preferable for the third party to disturb driver return todriving, and in a case where a disturbing action is performed, also itis necessary to leave a record of this.

In a case where it is necessary for the driver to return from automaticdriving to manual driving, if a third party joining in a secondary taskthrough remote connection knows whether the situation is temporary orthe state continues, then the third party need not be subject to actionrestriction, and therefore, the convenience is improved. Here, plandisplay of the driver return necessity in automatic driving by theautomatic driving vehicle may be notified, depending upon a use, furtherto a remote third party by remote communication.

In the present specification, in a case where, during a secondary taskexecuted by the driver described above, it is requested by the system totravel under attention along the traveling route or to return to manualdriving to transit to traveling, depending upon the contents of thesecondary task being executed, the requesting timing may not appropriateto interrupt the work of the secondary task, in some cases.

For example, in a case where the secondary task is a nap including deepsleep, there is a case in which a timing at which the sleep is soshallow that it is suitable to return to some degree can be decidedthrough steady state observation while the system normally andcontinuously observes the driver state such as depth decision of thesleep. However, in various kinds of secondary tasks the driver can take,it is very difficult to always calculate a notification timing optimumto the secondary task performed by the driver only from biologicalobservation information of the driver and so forth that can be performedby the system.

Depending upon the type of a secondary task, there is a type ofconsciousness withdrawal different from the withdrawal from an extremesteering work like the sleep described above. Also an immersive game isa typical example of such a secondary task as just described. Matter ofcourse, depending upon the way of involvement in the secondary task, thework may have the possibility that, even if the driver can start thework while paying attention forwardly, which is necessary for drivingreturn, simultaneously depending upon the way of the involvement anddepending upon the brain thinking activity, the driver may immersehimself/herself in the secondary task until the attention of the driverto the front of traveling or to a takeover notification timing degradessignificantly. If the driver reaches extreme addiction like gameaddiction, then the driver becomes insensitive to a notification or analert by the system, resulting in the possibility that appropriatereturn may not be achieved.

Further, a task that provides a similar immersive sense includeswatching of sports on live television or the like. Further, it isassumed that, during a period that involves lively discussion in atelephone conference or the like, depending upon a participationsituation in the discussion, the driver becomes insensitive similarly toa return notification or an alert by the system not in the conference.

Also a work for a slip inputting system to a tablet terminal apparatuscan be classified to a secondary task of the type in which it isgenerally favorable that, from a point of view of the work efficiency, aseries of inputting is performed continuously without interrupting thework and the inputting work is performed until the work settles down.

It is to be noted that, as a work that provides a milder immersivesense, for example, movie watching of a recorded video and viewing ofrecorded current news are available, and a work of any of thosesecondary tasks increases or decreases excessive attention concentrationto the secondary task depending upon the content. Depending upon thecurrent reproduction contents, the assignment of attention relating tosurrounding traveling sometimes becomes neglected, or sufficientattention maintenance can sometimes be achieved, in some cases.

However, it is generally difficult to take a correlation to the depth ofwithdrawal from driving attention by a work. Therefore, even if passiveobservation of the driver state by the system can be performed, sincethe system cannot observe up to a thinking situation in the brain,direct observation of the attention concentration level cannot beperformed, and it is difficult to achieve optimization of returnnotification or an alert timing.

In order to overcome this problem, there is no choice to rely uponvoluntary work interruption to some degree by the driver. In order tofacilitate interruption of a secondary task by the driver, it isnecessary to perform reduction of factors that give rise to hesitationto interrupt a secondary task.

In the several examples given hereinabove, since the driverhimself/herself cannot necessarily perform control of a degree of theprogress of a reproduction video or a game, the hesitation to interruptcan be reduced depending upon how the driver can comfortably watch thecontents of the secondary task continued from the interruption pointwhen the secondary task is interrupted and reproduction is restartedfrom the interruption point later.

In movie watching, live sports watching on live television, or the like,if appreciation of a movie or watching of sports on live television istemporarily interrupted at a good interruption scene such as a scenenext to a scene with which the emotional tension upon viewing rises andit is possible to restart appreciating or watching from the interruptionpoint, an unsatisfactory or discomfort feeling by the intermediateinterrupt can be reduced.

Further, at a point of time at which it becomes possible to restartappreciation, by providing supplementary information to the driver as aviewer upon restarting after insertion of a short summary of a story upto the interruption point or after insertion, of a highlight sceneduring the interruption in the case of sports live watching, it ispossible to achieve reduction of unsatisfactory feeling that occurs uponsecondary task interruption.

Such interruption of a secondary task for performing driving steeringattention and actual steering return with the secondary task interruptedmay be performed by an interruption procedure by the driver. Then, atthis time, restart from the interruption point may be performed or thereproduction point may be reserved retroactively in advance so as toperform interruption. The interruption reproduction method is desirablyinputted by such a method that, for example, an interruption menu isprovided by single touch an appreciation monitoring screen and intuitiveand rapid designation is possible such as interruption reproductionmethod designation or slider designation interruption of a retroactivepoint.

Further, in a case where specifically there is no designation,retroactive reproduction from the interruption point may be performed.Without retroactive reproduction, unlike continuous appreciation, sinceappreciation has been once interrupted, even if the driver watches thecontents from the interruption point upon restarting the reproduction ofthe contents, it is difficult to grasp the story of the contents. Theeffect of the present reproduction method can be recognized readily ifit is taken as an example that, even if a person hears a witty commentin a comedy double act some times later after hearing a funny comment,this is not funny.

As a secondary task having a higher degree of freedom in interruption,an example such as a work like data inputting to a tablet terminal, apersonal computer, or the like or reading is available. In the case ofdata inputting, upon inputting of information into a table or the likein an interlocking relationship while several items associated with eachother are checked, if the work is interrupted at a place that is notgood to stop, the work that has already been done at that time maypossibly become useless. Further, for inputting works for purchasethrough the Internet or official procedure inputting processes usedoften in various fields in recent years, a fixed continuous work isdemanded frequently. Therefore, in a case where an inputting work forsuch procedure cannot be ended, if the work is interrupted on the wayand the inputting work must be retroactively returned to the initialinput start point upon restarting of the inputting work, it is hesitatedto interrupt the inputting work.

As indicated by the example described above, relying also upon theimmersion degree in the secondary task, the necessity for continuousinputting, and so forth, if a task is of the type that the driver needsto redo the task all over again from the beginning in a case where thesecondary task is interrupted, even if the notification is received, thedriver psychologically wants to delay the interruption of the secondarytask to finish the work to a place that is good to stop, in some cases.

It is possible to cause the user to interrupt the secondary task in thefirst priority and encourage the user to return to the driving steeringwork at an early stage, when the disadvantage in a case where the workis interrupted at an early stage to take over the manual driving in thefirst priority is lower than the disadvantage in a case where the workof the secondary task is not interrupted and continued and the takeoverto the manual driving is delayed. However, as the disadvantage in a casewhere the work is not interrupted and continued thereby to cause delayof the takeover, a probability that delay of the driving return occursfinally may increase. In addition, even when the takeover is not carriedout in time as a result of delay on a rare case and the system causesthe vehicle to travel escaping travel for emergency, for example, unlessthe user recognizes the effect caused by the escaping travel asintuitively disadvantageous, the user does not try to avoid thesituation.

As a countermeasure for this, in a case where the system observes thatthe work is interrupted and the return is delayed, a mechanism forimposing a penalty on the user is effective. However, the penalty in acase where the return procedure delays is not always almighty, and asbehavioral psychology of a human, the user preferentially keeps thenotification of return until the user feels a direct penalty and thepsychology to interrupt the secondary task does not necessarily work.

Especially, if emergency slow traveling or escape traveling for riskminimization (Minimum Risk Maneuver) by the system is performed upongeneration of an urgent takeover request or when takeover is notcompleted in a supposed period of time, this forces peripheral travelingvehicles such as a subsequent vehicle to perform sudden braking oravoidance action, thereby causing an increase of risk such as a trafficjam or a rear-end accident. In particular, operation that relies onemergency measures performed by the system when a secondary task isforcibly continued unconditionally increases the probability that asecondary damage such as a traffic jam or rear-end accident may becaused and has an adverse effect that social functional degradation ofthe road infrastructure is caused.

A usage form is desirable which avoids reckless continuation of asecondary task at an early stage by a driver and encourages the driverto interrupt the secondary task quickly and start the takeover. Inparticular, if a penalty is generated against a violation of neglectinga motivation (incentive) and a request for early takeover, it isexpected that positive, voluntary and reflective early takeover of theuser becomes a habit, and also it becomes necessary to reduce demeritsof cumbersomeness in this case.

In particular, while the human psychology in automatic drivingutilization behavior becomes likely to rely upon automatic drivingsteering by the system in many traveling environments, a mechanism forprompting a voluntary return behavior to avoid laziness in performanceof takeover due to over-dependence is effective, and if the balancebetween advantages and disadvantages upon utilization appearsintuitively in operation feeling, a person starts a preferentialinterruption work.

As an example of a work for which continuous execution of a secondarytask is supposed, movie watching, sports watching, a board game or anelectronic game, conversation between passengers, discussion in atelephone conference, data inputting using an information terminal, anet banking work, texting of a mail or the like, browsing, net shopping,and so forth are available in the future.

Especially, among such secondary tasks as mentioned above, if a work isinterrupted in the middle of a game, further, in the middle of a work ina case where a series of information is inputted for an inputtingprocess of slips using an application with an information terminal suchas a smartphone or a tablet, or in a case of net shopping, the inputtingprocessing work which has been done so far all becomes wasted. As aresult, a situation in which the work must be performed from thebeginning (redone) possibly occurs.

Since it is desirable to eliminate wastefulness of a work as humanpsychology, if the psychology to complete inputting to the last works,interruption of the work is postponed, and the psychology that a littledelay may be permissible works further, resulting in a risk that safeand smooth return cannot be performed after all and may not be performedin time. In other words, as long as the driver is one human and atakeover work is performed in accordance with the human behavioralpsychology, a mechanism is required which interrupts a work in thebehavioral psychology and prioritizes early return.

Therefore, if a mechanism that interrupts a work and prioritizes earlyreturn can be constructed, then it is expected to reduce the risk bycausing the user (driver) to give up continuation of the work.Especially if the secondary task is information inputting on a tablet orthe like and an explicit menu that facilitates work designation pointreturn by the user on an execution application is prepared, then even ifthe user temporarily interrupts the work, the user can easily restartthe work from the interruption point.

It is to be noted that application software for many personal computersand other information terminals that are popular at present is equippedwith storage and recovery functions for a history of executionprocessing used for cancellation or redoing of inputting. However, thosefunctions are supposed for utilization for the purpose of redoing ofinputting during engagement in inputting on a terminal and are selectivereflection of selectively changed contents during a review work of adocument worked by a plurality of persons, but are not functions for theobject of assistance in restarting of an inputting work when anunspecified arbitrary inputting work is interrupted. Therefore, thefunctions become a factor for causing the user to hesitate interruptionof an inputting work to an information terminal as a secondary task.

In order for the user to interrupt an inputting work and preferentiallyreturn to a driving task, an assistive function for assisting restart ofthe inputting work or the like is demanded. In the following, workingexamples for assisting such return are depicted together with severaluses.

In the case of performance of a secondary task utilizing an informationterminal, in order to avoid full withdrawal from driving attention by anoversight in a return notification from automatic driving because ofexcessive immersion of the user in a secondary task, in a work window400, index presentation (traveling section display image 200) tillarrival at a takeover point by progress is performed normally. Further,if it is decided by the takeover notification decision device that it isa notification timing, a small icon for notification is displayed as avisual stimulus induction by flickering or the like in the work window400 as depicted in FIG. 20(a).

In a case where the driver (secondary task performer) acknowledges thenotification and performs, for example, touch with or check mark in theicon, for example, as depicted in FIG. 20(b), the work screen isinterrupted (partially obstructively) and a restart return pointdesignation slider menu for an inputting work is displayed such thatfree setting can be performed by slider rotation of an optimum returnpoint by the utilizer. In this case, the slider display may be arotational display image of the rotation type in the counterclockwisedirection as depicted in FIG. 20(b), a horizontal linear display imageas depicted in FIG. 21(a) or a vertical linear display image notdepicted.

In the case of such a clockwise display image as depicted in FIG. 20(b),the counterclockwise direction to the nine o'clock direction from the 12o'clock direction is determined as an input retroactive point, and inthe case of such a horizontal linear slider display form as depicted inFIG. 21(a), for example, the leftward direction from a current inputpoint that is the point at a length of approximately ⅔ is determined asan input retroactive point.

In the case of an information inputting process, the last inputinformation point is the current point, and since a place not inputtedas yet is not a point at which restart cannot be performed originally,it cannot be designated as a reproduction designation point uponrestarting. However, by progressing work items on a menu, the inputpoint is progressed to a planned inputting place on an inputtingapplication tool, and by executing screen simulation simple display uponreturn inputting, an optimum point determination for restarting the workretroactively can be performed.

Since the reason is a little difficult to understand, a descriptionthereof will be given taking high-jump in sports as an example. If thedistance to be jumped over upon high-jumping can be estimated inadvance, it can be determined what distance is to be assured as adistance for running. Therefore, although the running is interruptedonce and the jump is performed later, by grasping the situation inadvance, it can be predicted at which retroactive point the running isto be restarted. A slider design from which an input place in the futurecan be postponed and browsed is measures for this. This merit is usefulin a case where the remaining input items are checked.

In a case where fixed time lapse transition or acknowledgment responseinputting of the driver to the notification is not performed in responseto the notification and the system fails in confirmation of responsedetection of the driver, it is reasonable to issue a warning to promptearly return. For example, index presentation (traveling section displayimage 200) till arrival at a takeover point in the moving direction isdisplayed in an enlarged scale like a traveling section display image200′ as depicted in FIG. 21(b) while the secondary task execution window400 is reduced, and in some cases, the application inputting beingexecuted as a secondary task may be force-quit.

In a case where the driver who is executing a secondary task designatesa work restart point and interrupts the work once in response to areturn request by the system and the driver can return to automaticdriving again as a flow and restarts the work, as depicted in FIG. 22, aslider menu (slider for return place designation with an input historyon the screen upon restarting) 500 in which input work points (a CR(enter keyboard Carriage Return) execution point to an input work pointand an input form completion point) are lined up explicitly inchronological order may be presented. It is to be noted that, althoughthe restart after interruption of the work once need not necessarily beperformed after return to automatic driving and may be performed afterthe end of the itinerary, an example of the applicable case is omittedin the flow chart.

Especially, if there is no necessity to designate a particular restartpoint, the slider may be closed by a process for slidably moving theslider with double fingers or by a close menu. Alternatively, a restartpoint may be designated by inputting a numerical value, hitting CarriageReturn, moving the slider to a location at which work restarting isdesired to be performed among a plurality of points and designating theapplicable location by double touch or checkmark operation.

These designation of the return point and the operation are made tocoincide with each other, and the designation location is moved on theslider, so that it is also effective to reproduce an input screen on anapplication execution screen. Also upon restarting of the work, bydisplaying a work progress menu partitioned for each input return key ina slider, the inputting person can easily check an interruption placeand an input history that has been done so far, and therefore, thisbecomes an interface by which the interruption point can be recalledreadily.

Especially, since an input work point of a terminal work and a reachpoint of a traveling environment on a tablet or the like can be graspedintuitively by the user, not only an execution task of a secondary taskand display update information are temporarily displayed simply to thedriver but also a synchronous record with the task may be further storedinto a recording apparatus such that the task playback function and soforth can be used together by a scroll of a section approach windowinvolved in the progress on the menu and a screen double touch or tripletouch operation with a work recovery point and so forth.

By providing means for work return point search and return markers withwhich return of the work to an arbitrary work point is readily possibleafter driving return to the driver in this manner, reckless continuationof a secondary task can be avoided from an ergonomic perspective. As aresult, since the driver who is executing the secondary task startsdriving return quickly without stress, this leads to achievement ofutilization of safer automatic driving can be implemented.

Especially, in an inputting work, an interaction between preceding andsucceeding pieces of input information is important, and for example, ina case where a plurality of pieces of related information aresuccessively inputted into a table or the like, in a case where an itemis selected and a series of information interlocked with the item are tobe inputted, if inputting is interrupted at a place that is not good tostop under an incomplete condition, depending upon the memory of thedriver upon restart the work, there is the possibility that time may berequired to recall the interruption place or incorrect input may occur.

In particular, in a case where an application interface that presupposesordinary continuous use remains as it is, when an inputting work isinterrupted on the way and a work with attention different from theinputting work required is performed during the interruption, there isan adverse effect when the driver returns to the interruption placeprecisely upon work return and continues the work. By performingexplicit visualization of a menu for exclusive use for work interruptionpoint designation or of an inputting location, upon work return, thedriver recalls the memory of the inputting work to allow for easy workrestart.

The input screen change history till the time of interruption ofinputting a secondary task is an effective auxiliary function forrecalling an input restart point and is useful in improvement of thework efficiency. Especially in a work in which a browsing effect ofhistory information is high but is heavy in recalculation, it is alsopossible to omit the re-calculation process by locally storing screenchanges as images over last several stages.

Although the example described above is a flow as viewed from a point ofview of data inputting to an information terminal principally as aninformation inputting application, a similar operation flow isapplicable also to movie watching, sports watching and streaming viewingof a news program. In the case of movie watching or watching acompetition game such as the soccer which is being broadcasted on atelevision, it is effective to temporarily and locally store thebroadcast, interrupt reproduction display at a timing at which manualdriving return is requested once and perform delayed reproduction fromthe middle of the work by the driver. At this time, it is possible tolower the necessity for the cumbersome rewound playback process.

However, in a case where an extreme utilization form such asparticularly VR utilization in a participatory game that provides asense of deep immersion is performed, also it is effective to not onlyforce the driver to interrupt the work but also force the driver to viewforward traveling information by switching display of the screen suchthat the forward traveling information is displayed in the secondarytask screen, causing the driver to return his/her attention from a statein which the driver is fully withdrawn from the driving traveling work,at an early stage.

In a case where the advantages obtained by interrupting a secondary taskwork during automatic driving can be felt in the short term only insafety and reliable and smooth takeover at a takeover limit point, thenecessity for the same is not felt intuitively. Therefore, although astepwise penalty function is effective to cause the user to feel thenecessity for start of intuitive return and to prompt an action, alertusing an alarm or haptics is cumbersome and punishment only cannotalways improve the human behavioral psychology. However, by combining amechanism for supporting restart and return upon the secondary taskinterruption that cancels the disadvantages, it is possible to grow thepsychology of rapid interruption of the secondary task and habituallyreduce reckless continuation of the secondary task.

Flow charts of FIGS. 23 and 24 depict an example of a processingprocedure of the system in a case where a takeover notification decisionis received.

At step S51, the system starts processing when it receives a takeovernotification decision. Then, at step S52, the system executesnotification to a performer of a secondary task. For example, in a casewhere a secondary task in which an information terminal is used isperformed, a small icon for notification is flickered or the like in awork window 400 (refer to FIG. 20(a)).

Then at step S53, the system detects notification acknowledgementresponse of the performer of the secondary task. For example, in a casewhere the secondary task utilizing an information terminal is performedand a small icon for notification is flickered or the like as anotification in the work window 400 as described above, for example, atouch operation or the like by the performer with the icon is detected.Further, at step S54, the system issues an interruption request of theinputting work of the secondary task to the performer of the secondarytask and gives a notice for screen reduction and interruption.

Then at step S55, the system discriminates whether or not a notificationacknowledgement response is detected. If a notification acknowledgementresponse is not detected, the system calculates a remaining time periodfor a takeover grace budget at step S56. Then, at step S57, the systemdiscriminates on the basis of the remaining time period calculated atstep S56 whether or not it is a timeout. If it is not a timeout, thesystem returns to step S53 to repeat processes similar to thosedescribed above.

In contrast, if it is a timeout at step S57, the system executes warningusing sound, vibration, an image, or the like at step S58. Then, at stepS59, the system performs reduction of a work screen image andenlargement emphasis display of a takeover point approach display (referto FIG. 21(b)). Then, at step S60, the system performs an acceptanceprocess of warning acknowledgment and restart point designation of thesecondary task. In this case, the system displays a restart pointdesignation slider menu for the secondary task inputting work andperforms acceptance of determination of the restart point designation(work return point reservation by a slider or the like) by the secondarytask performer (refer to FIGS. 20(b) and 21(a)).

Then, at step S61, the system discriminates whether or not it is atimeout. In this case, the timeout is discriminated at a point of timeat which takeover delay is predicted by the takeover grace time perioddecision. If it is not a timeout, the system continues the process atstep S60. In contrast, if it is a timeout, the system forcibly ends thework of the secondary task at step S62. Then at step S63, the systemexecutes return delay penalty recording, restriction of the work restartpoint, reuse restriction of the automatic driving mode and so forth.

Conversely, when a notification acknowledgement response is detected atstep S55 described above, the system displays a restart pointdesignation slider menu for the secondary task inputting work at stepS64 and then performs acceptance of determination of the restart pointdesignation (work return point reservation by a slider or the like) bythe secondary task performer (refer to FIGS. 20(b) and 21(a)). Then,after the process at step S65, the system advances to a process at stepS66. Also after the process at step S63 described hereinabove, thesystem advances to the process at step S66.

At step S66, the system saves a work history and the restart pointdesignation information of the secondary task. Then, at step S67, thesystem decides that it is a manual driving return period and executionof the secondary task is disabled.

Then, at step S68, the system monitors and acquires LDM information,driver states and vehicle self-diagnosis information necessary forautomatic driving restart availability decision. Then, at step S69, thesystem determines whether or not the vehicle has entered again into anexecutable section of the secondary task. If the vehicle has not enteredagain the executable section of the secondary task, the system continuesthe process at step S68. In contrast, if the vehicle has entered againthe executable section of the secondary task, the system determinesautomatic driving available section entry at step S70.

Then, at step S71, the system enables a returnable menu of a secondarytask work of the terminal. Then, at step S72, the system monitors asecondary task return restart input. Then, at step S73, the systemdetermines whether or not selection of the return menu is performed bythe performer of the secondary task. If selection of the return menu isnot performed, the system determines at step S74 whether or not it is atimeout (restart designation waiting time timeout). If it is not a timeout, the system returns to the process at step S71 to repeat processessimilar to those described hereinabove. The timeout decision performedat step S74 may be a procedure that, in a case where the driver does notperform secondary task restart within a fixed period of time,interruption of the work equivalent to transition to a standby mode thatis an energy saving mode of a terminal or the like, a pause mode inwhich interruption information for turning off the power supply fully isrecorded to close the work or the like is performed such that loss ofthe work contents that have been done so far is avoided by manualdriving return that requires long-term work interruption.

When the selection of the return menu is performed at step S73, if thework restart point designation is performed through a slider menu or thelike by the system at step S75, at step S76, the system performs workrestart in accordance with the designation input. Thereafter, the systemends the series of processes at step S77. It is to be noted that, whenit is a timeout at step S74, the system immediately advances to step S77to end the series of processes.

[Modification 4]

Further, in the present specification, in a case where, during asecondary task executed by the driver as described above, the secondarytask is interrupted on the basis of traveling under caution along atraveling route or manual driving return request information toprioritize manual driving return, there possibly is a case in which thetiming at which return is demanded to the driver not necessarily is atiming that is good for interruption of the secondary task beingexecuted. Especially, there possibly is a situation in which, in a casewhere a series of information is to be inputted, if the work isinterrupted on the way, then the inputting processing work that has beendone so far must all be re-inputted from the beginning. Since, as humanpsychology, it is desired to eliminate wastefulness of a work, it istried to finish inputting to the last. However, this involves a riskthat return is not performed in time. Therefore, as risk reductionmeasures, an explicit menu that facilitates work designation pointreturn by the user on an execution application of a secondary task mayfurther be provided auxiliarily. Especially, since an inputting workpoint of a terminal work on a tablet or the like and a reaching point ofa traveling environment can be grasped intuitively by the user, not onlyan execution task of the secondary task and display update informationare temporarily presented to the driver simply such that a task playbackfunction and so forth can be used by both of a scroll of a sectionapproach window together with traveling on a menu and a screen doubletouch or triple touch operation of a work recovery point or the like butalso synchronous recording with the task may be further stored into arecording apparatus. By providing means returnable to an arbitrary workpoint after driving return to the driver in this manner, recklesscontinuation of the secondary task can be avoided from an ergonomicpoint of view. As a result, the driver who is executing the secondarytask starts driving return quickly without a stress, and this leads toimplementation of utilization of safer automatic driving.

[Modification 5]

Further, in the case of traveling support in which remote support bywireless communication is received in operation in a line traveling, aremote supporter may control the traveling speed at a support requiringpoint for the applicable vehicle or vehicle group to perform plannedallocation or perform distribution planning avoiding concentration ofsupport of the traffic control center. Further, the traveling supportmay be utilized for planning or the like of building a distribution planbetween an automatic delivery partner and a manual delivery partner by adelivery company, and utilization of the traveling support may befurther expanded to delivery remote support and so forth.

[Modification 6]

Further, the embodiment of the present technology is not limited to theembodiment described hereinabove and can be modified in various mannerswithout departing from the subject matter of the present technology. Forexample, the present technology can take a configuration for cloudcomputing by which one function is shared and cooperatively processed bya plurality of apparatuses through a network.

Further, the steps described in the description of the flow charts givenhereinabove not only can be executed by a single apparatus but also canbe shared and executed by a plurality of apparatuses. Further, in a casewhere one step includes a plurality of processes, the plurality ofprocesses included in the one step can be executed by a single apparatusand also can be executed by sharing by a plurality of apparatuses.

It is to be noted that the present technology can take also suchconfigurations as described below.

(1) An information processing apparatus including:

an information acquisition unit configured to acquire traveling routeinformation and traffic information relating to the traveling route; and

a display controlling unit configured to display a driver interventionrequiring section and an automatic driving available section of thetraveling route on a reach prediction time axis from a current point ona display device on the basis of the traveling route information and thetraffic information.

(2) The information processing apparatus according to (1) above, inwhich

the driver intervention requiring section includes a manual drivingsection, a takeover section from automatic driving to manual driving,and a cautious traveling section from the automatic driving.

(3) The information processing apparatus according to (2) above, inwhich

the display controlling unit displays the automatic driving availablesection in a first color, displays the manual driving section in asecond color, and displays the takeover section and the cautioustraveling section in a third color.

(4) The information processing apparatus according to any one of (1) to(3) above, in which

the display controlling unit

displays a first section from the current point to a first point on afirst time axis,

displays a second section from the first point to a second point in atime axis that sequentially changes from the first time axis to a secondtime axis reduced at a predetermined ratio with respect to the firsttime axis, and

displays a third section from the second point to a third point on thesecond time axis.

(5) The information processing apparatus according to (4) above, inwhich

the display controlling unit

displays the first section with a first width,

displays the second section with a width that sequentially changes fromthe first width to a second width smaller than the first width, and

displays the third section with the second width.

(6) The information processing apparatus according to (4) above, inwhich

the display controlling unit displays the driving vehicle interventionrequiring section in the third section with a fixed time length even ifthe driving vehicle intervention requiring section actually is equal toor smaller than the fixed time length.

(7) The information processing apparatus according to any one of (1) to(6) above, in which

the display controlling unit further displays information relating to apoint designated in each of the displayed sections.

(8) The information processing apparatus according to any one of (1) to(7) above, in which

the display controlling unit displays the driving vehicle interventionrequiring section that appears newly so as to be identifiable from theexisting driving vehicle intervention requiring section.

(9) The information processing apparatus according to any one of (1) to(8) above, in which

when the driver intervention requiring section comes in a range of afixed period of time from the current point, the display controllingunit puts the driver intervention requiring section into an emphaticallydisplayed state.

(10) The information processing apparatus according to any one of (1) to(9) above, in which

the display controlling unit displays display images in the sections inparallel to a work window.

(11) The information processing apparatus according to any one of (1) to(10) above,

the display device being a display device provided in a portableterminal, and

the information processing apparatus further including:

a communication unit configured to communicate with the portableterminal.

(12) An information processing method including:

by an information acquisition unit, a step of acquiring traveling routeinformation and traffic information relating to the traveling route; and

by a display controlling unit, a step of displaying a driverintervention requiring section and an automatic driving availablesection of the traveling route on a reach prediction time axis from acurrent point on a display device on the basis of the traveling routeinformation and the traffic information.

REFERENCE SIGNS LIST

-   -   100 . . . Vehicle controlling system    -   101 . . . Inputting unit    -   102 . . . Data acquisition unit    -   103 . . . Communication unit    -   104 . . . In-vehicle apparatus    -   105 . . . Output controlling unit    -   106 . . . Outputting unit    -   107 . . . Drive-train system controlling unit    -   108 . . . Drive-train system    -   109 . . . Body controlling unit    -   110 . . . Body system    -   111 . . . Storage unit    -   112 . . . Automatic driving controlling unit    -   121 . . . Communication network    -   131 . . . Detection unit    -   132 . . . Self-position estimation unit    -   133 . . . Situation analysis unit    -   134 . . . Planning unit    -   135 . . . Motion controlling unit    -   141 . . . Outside-vehicle information detection unit    -   142 . . . In-vehicle information detection unit    -   143 . . . Vehicle state detection unit    -   151 . . . Map analysis unit    -   152 . . . Traffic rule recognition unit    -   153 . . . Situation recognition unit    -   154 . . . Situation prediction unit    -   161 . . . Route planning unit    -   162 . . . Action planning unit    -   163 . . . Motion planning unit    -   171 . . . Emergency avoidance unit    -   172 . . . Acceleration/deceleration controlling unit    -   173 . . . Direction controlling unit

1. An information processing apparatus comprising: an informationacquisition unit configured to acquire traveling route information andtraffic information relating to the traveling route; and a displaycontrolling unit configured to display a driver intervention requiringsection and an automatic driving available section of the travelingroute on a reach prediction time axis from a current point on a displaydevice on a basis of the traveling route information and the trafficinformation.
 2. The information processing apparatus according to claim1, wherein the driver intervention requiring section includes a manualdriving section, a takeover section from automatic driving to manualdriving, and a cautious traveling section from the automatic driving. 3.The information processing apparatus according to claim 2, wherein thedisplay controlling unit displays the automatic driving availablesection in a first color, displays the manual driving section in asecond color, and displays the takeover section and the cautioustraveling section in a third color.
 4. The information processingapparatus according claim 1, wherein the display controlling unitdisplays a first section from the current point to a first point on afirst time axis, displays a second section from the first point to asecond point in a time axis that sequentially changes from the firsttime axis to a second time axis reduced at a predetermined ratio withrespect to the first time axis, and displays a third section from thesecond point to a third point on the second time axis.
 5. Theinformation processing apparatus according to claim 4, wherein thedisplay controlling unit displays the first section with a first width,displays the second section with a width that sequentially changes fromthe first width to a second width smaller than the first width, anddisplays the third section with the second width.
 6. The informationprocessing apparatus according to claim 4, wherein the displaycontrolling unit displays the driving vehicle intervention requiringsection in the third section with a fixed time length even if thedriving vehicle intervention requiring section actually is equal to orsmaller than the fixed time length.
 7. The information processingapparatus according to claim 1, wherein the display controlling unitfurther displays information relating to a point designated in each ofthe displayed sections.
 8. The information processing apparatusaccording to claim 1, wherein the display controlling unit displays thedriving vehicle intervention requiring section that appears newly so asto be identifiable from the existing driving vehicle interventionrequiring section.
 9. The information processing apparatus according toclaim 1, wherein when the driver intervention requiring section comes ina range of a fixed period of time from the current point, the displaycontrolling unit puts the driver intervention requiring section into anemphatically displayed state.
 10. The information processing apparatusaccording to claim 1, wherein the display controlling unit displaysdisplay images in the sections in parallel to a work window.
 11. Theinformation processing apparatus according to claim 1, the displaydevice being a display device provided in a portable terminal, and theinformation processing apparatus further comprising: a communicationunit configured to communicate with the portable terminal.
 12. Aninformation processing method comprising: by an information acquisitionunit, a step of acquiring traveling route information and trafficinformation relating to the traveling route; and by a displaycontrolling unit, a step of displaying a driver intervention requiringsection and an automatic driving available section of the travelingroute on a reach prediction time axis from a current point on a displaydevice on a basis of the traveling route information and the trafficinformation.